2024-09-10T09:43:07.677 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T09:43:07.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T09:43:07.809 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} email: null first_in_suite: false job_id: '7899009' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHLwiwPhXk3nm4DLyD2wIl9YiUHe/5BvaT1UQMHFmtMPZ7CnrXMC+oEwdNhMBwMnPYHHEfwUcmi6lpptNuX2DHE= smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN9HUtdrzyvacXhkx0BFWWPrA5l/k9gMEvRtDHnQeS46z6h3Zb1cwEUSsOijyFPIozk+b/9VTpPogpsnIKfCnwg= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5ETa+b/JJiQXvht9vZC1dR3p0c9rMnuJpk1HpQbzslXjF618ZincQyP0UMxgJksNvpkB0x7qpSC+yWdTMfp9U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T09:43:07.809 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T09:43:07.810 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T09:43:07.810 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T09:43:07.812 INFO:teuthology.task.internal:Checking packages... 2024-09-10T09:43:07.834 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T09:43:07.834 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T09:43:07.834 INFO:teuthology.packaging:ref: None 2024-09-10T09:43:07.834 INFO:teuthology.packaging:tag: None 2024-09-10T09:43:07.834 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:43:07.834 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:43:07.835 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:43:07.975 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T09:43:07.977 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T09:43:07.984 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T09:43:07.985 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T09:43:07.991 INFO:teuthology.task.internal:Saving configuration 2024-09-10T09:43:08.012 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T09:43:08.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T09:43:08.042 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:38:07.574348', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHLwiwPhXk3nm4DLyD2wIl9YiUHe/5BvaT1UQMHFmtMPZ7CnrXMC+oEwdNhMBwMnPYHHEfwUcmi6lpptNuX2DHE='} 2024-09-10T09:43:08.063 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:38:07.573121', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN9HUtdrzyvacXhkx0BFWWPrA5l/k9gMEvRtDHnQeS46z6h3Zb1cwEUSsOijyFPIozk+b/9VTpPogpsnIKfCnwg='} 2024-09-10T09:43:08.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:38:07.575333', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL5ETa+b/JJiQXvht9vZC1dR3p0c9rMnuJpk1HpQbzslXjF618ZincQyP0UMxgJksNvpkB0x7qpSC+yWdTMfp9U='} 2024-09-10T09:43:08.085 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T09:43:08.093 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T09:43:08.093 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T09:43:08.093 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T09:43:08.093 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T09:43:08.178 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8da41876d0>, signals=[15]) 2024-09-10T09:43:08.178 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T09:43:08.184 INFO:teuthology.task.internal:Opening connections... 2024-09-10T09:43:08.184 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T09:43:08.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:43:08.258 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-09-10T09:43:08.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:43:08.332 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-09-10T09:43:08.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:43:08.407 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T09:43:08.413 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-09-10T09:43:08.432 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-09-10T09:43:08.432 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-09-10T09:43:08.489 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-09-10T09:43:08.489 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-09-10T09:43:08.489 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-09-10T09:43:08.489 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:43:08.490 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:43:08.491 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-09-10T09:43:08.506 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-09-10T09:43:08.525 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-09-10T09:43:08.525 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-09-10T09:43:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:43:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:43:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:43:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:43:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:43:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:43:08.584 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-09-10T09:43:08.598 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-09-10T09:43:08.616 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-09-10T09:43:08.616 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-09-10T09:43:08.675 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-09-10T09:43:08.675 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-09-10T09:43:08.675 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-09-10T09:43:08.675 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:43:08.675 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-09-10T09:43:08.675 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:43:08.676 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:43:08.676 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-09-10T09:43:08.698 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T09:43:08.707 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T09:43:08.713 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T09:43:08.713 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:43:08.716 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:43:08.718 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:43:08.736 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T09:43:08.742 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T09:43:08.743 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:43:08.773 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:43:08.775 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:43:08.800 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T09:43:08.877 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T09:43:08.891 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T09:43:08.892 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-10T09:43:08.892 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-10T09:43:08.892 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:43:08.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:43:08.910 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-10T09:43:08.968 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-503.el9.x86_64 2024-09-10T09:43:08.969 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-503.el9.x86_64 2024-09-10T09:43:08.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-09-10T09:43:11.830 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.2 MB 00:01 2024-09-10T09:43:13.781 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-10T09:43:19.690 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:01 2024-09-10T09:43:21.156 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 87 kB/s | 19 kB 00:00 2024-09-10T09:43:21.944 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-10T09:43:27.658 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T09:43:29.748 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T09:43:29.749 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T09:43:29.839 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-10T09:43:29.843 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-10T09:43:29.843 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-09-10T09:43:29.844 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:43:29.845 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 79 M 2024-09-10T09:43:29.845 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 126 M 2024-09-10T09:43:29.845 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-10T09:43:30.592 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 40 kB/s | 23 kB 00:00 2024-09-10T09:43:31.551 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-10T09:43:32.134 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-10T09:43:32.885 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-10T09:43:32.885 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:43:32.885 INFO:teuthology.orchestra.run.smithi045.stdout:Total 26 MB/s | 79 MB 00:03 2024-09-10T09:43:33.601 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-10T09:43:33.976 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-10T09:43:33.976 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-10T09:43:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-10T09:43:34.944 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-10T09:43:37.544 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-10T09:43:38.213 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T09:43:38.312 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T09:43:41.108 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T09:43:41.239 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T09:43:46.370 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T09:43:46.415 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T09:43:51.562 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T09:44:30.569 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T09:44:30.610 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T09:44:30.911 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T09:44:30.911 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T09:44:30.912 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T09:44:30.912 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T09:44:31.754 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:44:31.755 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-10T09:44:32.123 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T09:44:32.254 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T09:44:32.254 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T09:44:32.254 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T09:44:32.254 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T09:44:32.255 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T09:44:32.255 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T09:44:32.255 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T09:44:32.255 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T09:44:32.315 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T09:44:32.316 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T09:44:32.316 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-09-10T09:44:32.355 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-09-10T09:44:32.355 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-09-10T09:44:32.355 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-09-10T09:44:32.355 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 82.el9 2024-09-10T09:44:32.355 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-09-10T09:44:32.355 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T09:44:32.355 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8274670 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T09:44:32.356 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T09:44:32.357 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T09:44:32.357 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-09-10T09:44:32.357 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:44:32.357 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T09:44:32.358 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T09:44:32.358 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-505.el9.x86_64 2024-09-10T09:44:32.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T09:44:33.010 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-09-10T09:44:34.463 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T09:44:34.493 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-09-10T09:44:34.499 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-09-10T09:44:34.515 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.Wc5iPUhKxL 2024-09-10T09:44:34.515 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Wc5iPUhKxL 2024-09-10T09:44:34.587 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.Wc5iPUhKxL 2024-09-10T09:44:34.733 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.Wc5iPUhKxL is 6KB 2024-09-10T09:44:34.782 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.Wc5iPUhKxL 2024-09-10T09:44:34.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T09:44:34.859 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T09:44:34.859 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T09:44:34.859 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T09:44:34.859 INFO:teuthology.orchestra.run.smithi045.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T09:44:34.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T09:44:35.015 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-09-10T09:45:05.018 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T09:45:05.019 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-10T09:45:05.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:45:23.517 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-09-10T09:45:32.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-10T09:45:32.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:46:32.520 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T09:46:44.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-10T09:46:44.523 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:46:44.830 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-10T09:46:45.185 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-10T09:46:45.185 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T09:46:45.185 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-09-10T09:46:45.202 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-505.el9.x86_64 2024-09-10T09:46:45.202 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T09:46:45.202 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T09:46:45.202 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T09:46:46.204 DEBUG:teuthology.parallel:result is None 2024-09-10T09:46:46.204 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T09:46:46.212 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T09:46:46.212 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:46:46.215 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:46:46.217 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:46:46.236 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T09:46:46.243 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T09:46:46.252 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T09:46:46.252 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:46:46.273 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:46:46.275 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:46:46.309 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T09:46:46.315 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T09:46:46.315 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:46:46.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:46:46.355 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:46:46.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:46:46.371 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:46:46.374 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:46:46.397 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:46:46.434 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:46.454 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:46.459 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:46.459 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:46.475 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:46.476 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:46:46.478 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T09:46:46.485 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T09:46:46.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:46:46.488 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:46:46.498 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:46:46.526 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T09:46:46.535 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T09:46:46.535 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:46:46.559 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:46:46.566 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:46:46.584 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:46.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:46.709 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:46.798 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:46.843 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:46:46.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:46:46.913 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:46.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:47.014 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:47.099 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:47.143 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:46:47.143 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:46:47.212 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:47.258 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:46:47.317 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:47.399 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:46:47.445 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:46:47.445 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:46:47.514 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-09-10T09:46:47.516 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-09-10T09:46:47.519 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-09-10T09:46:47.557 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:46:47.566 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:46:47.600 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:46:47.947 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T09:46:47.958 INFO:teuthology.task.internal:Starting timer... 2024-09-10T09:46:47.958 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T09:46:48.023 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T09:46:48.031 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T09:46:48.031 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-09-10T09:46:48.102 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-09-10T09:46:48.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-09-10T09:46:48.173 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-09-10T09:46:48.174 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-09-10T09:46:48.234 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-09-10T09:46:48.236 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T09:46:48.236 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-09-10T09:46:48.260 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-09-10T09:46:48.260 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-09-10T09:46:48.280 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-09-10T09:46:48.280 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-09-10T09:46:48.303 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-09-10T09:46:48.303 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2024-09-10T09:46:48.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:46:48.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:46:48.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:46:48.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:46:48.362 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:46:48.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:46:48.391 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T09:46:48.392 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:46:48.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:46:48.481 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:46:48.517 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T09:46:48.526 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T09:46:48.529 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T09:46:48.648 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T09:46:48.658 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T09:46:48.659 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi047.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T09:50:51.159 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-09-10T09:50:51.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-10T09:50:51.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:50:51.238 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-09-10T09:50:51.317 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-09-10T09:50:51.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T09:50:51.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:50:51.391 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-09-10T09:50:51.473 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-09-10T09:50:51.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T09:50:51.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:50:51.547 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-10T09:50:51.632 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T09:50:51.633 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T09:50:51.644 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T09:50:51.644 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:50:51.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:50:51.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:50:51.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:50:51.650 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:50:51.678 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:50:51.680 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:50:51.693 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:50:51.696 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:50:51.702 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:50:51.717 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:50:51.718 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-09-10T09:50:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-09-10T09:50:51.730 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-09-10T09:50:51.736 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-09-10T09:50:51.742 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-09-10T09:50:51.744 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:50:51.751 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:50:51.753 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-09-10T09:50:51.758 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:50:51.767 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:50:51.768 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:50:51.782 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:50:51.813 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:50:51.817 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:50:51.817 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-10T09:50:51.817 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.817 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.817 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.817 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.826 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:50:51.829 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:50:51.829 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-10T09:50:51.830 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.830 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.830 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.830 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.843 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:50:51.846 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:50:51.846 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-10T09:50:51.846 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.846 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.846 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.846 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:50:51.847 INFO:teuthology.run_tasks:Running task install... 2024-09-10T09:50:51.858 DEBUG:teuthology.task.install:project ceph 2024-09-10T09:50:51.858 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T09:50:51.859 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T09:50:51.859 INFO:teuthology.task.install:Using flavor: default 2024-09-10T09:50:51.864 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T09:50:51.864 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T09:50:51.865 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:50:51.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:50:51.867 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:50:51.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:50:51.868 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:50:51.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:50:52.011 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:50:52.011 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:50:52.023 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:50:52.024 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:50:52.030 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:50:52.030 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:50:52.121 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:50:52.121 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:50:52.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:50:52.132 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:50:52.132 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:50:52.132 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:50:52.139 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:50:52.139 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:50:52.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:50:52.149 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:50:52.149 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:50:52.157 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:50:52.158 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:50:52.166 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:50:52.166 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:50:52.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:50:52.223 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:50:52.232 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:50:52.293 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:50:52.299 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:50:52.307 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:50:52.358 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-09-10T09:50:52.359 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-09-10T09:50:52.363 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-09-10T09:50:52.364 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-09-10T09:50:52.371 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-09-10T09:50:52.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-09-10T09:50:52.592 INFO:teuthology.orchestra.run.smithi162.stdout:45 files removed 2024-09-10T09:50:52.595 INFO:teuthology.orchestra.run.smithi045.stdout:45 files removed 2024-09-10T09:50:52.602 INFO:teuthology.orchestra.run.smithi047.stdout:45 files removed 2024-09-10T09:50:52.618 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:50:52.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:50:52.630 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:50:53.167 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-10T09:50:53.183 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-10T09:50:53.197 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-10T09:50:53.450 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T09:50:53.466 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T09:50:53.480 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T09:50:53.726 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T09:50:53.735 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-10T09:50:53.748 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T09:50:54.476 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-10T09:50:54.881 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.2 MB 00:01 2024-09-10T09:50:55.651 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-10T09:50:59.157 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-10T09:51:03.675 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-10T09:51:05.234 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-10T09:51:05.972 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-10T09:51:11.720 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T09:51:12.142 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:16 2024-09-10T09:51:14.166 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:51:14.167 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:51:14.182 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:51:14.183 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:51:14.184 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:51:14.185 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:51:14.187 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:51:14.190 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:51:14.191 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:51:14.258 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:51:14.277 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:51:14.278 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:51:14.279 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:51:14.280 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:51:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:51:14.282 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:51:14.283 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:51:14.284 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:51:14.285 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:51:14.286 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:51:14.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:51:14.288 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout:====================================================================================== 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 8 Packages 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 226 M 2024-09-10T09:51:14.289 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-09-10T09:51:14.746 INFO:teuthology.orchestra.run.smithi045.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-10T09:51:14.966 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-09-10T09:51:15.005 INFO:teuthology.orchestra.run.smithi045.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 3.4 MB/s | 897 kB 00:00 2024-09-10T09:51:15.097 INFO:teuthology.orchestra.run.smithi045.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.6 MB/s | 145 kB 00:00 2024-09-10T09:51:15.289 INFO:teuthology.orchestra.run.smithi045.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 8.0 MB/s | 5.4 MB 00:00 2024-09-10T09:51:15.547 INFO:teuthology.orchestra.run.smithi045.stdout:(5/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 23 MB/s | 21 MB 00:00 2024-09-10T09:51:15.589 INFO:teuthology.orchestra.run.smithi045.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 5.1 MB/s | 1.5 MB 00:00 2024-09-10T09:51:15.639 INFO:teuthology.orchestra.run.smithi045.stdout:(7/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 4.4 MB/s | 2.4 MB 00:00 2024-09-10T09:51:15.748 INFO:teuthology.orchestra.run.smithi045.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 24 MB/s | 4.7 MB 00:00 2024-09-10T09:51:15.790 INFO:teuthology.orchestra.run.smithi045.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 605 kB/s | 25 kB 00:00 2024-09-10T09:51:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 17 MB/s | 17 MB 00:01 2024-09-10T09:51:16.657 INFO:teuthology.orchestra.run.smithi045.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 4.5 MB/s | 228 kB 00:00 2024-09-10T09:51:16.724 INFO:teuthology.orchestra.run.smithi045.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 407 kB/s | 27 kB 00:00 2024-09-10T09:51:16.783 INFO:teuthology.orchestra.run.smithi045.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 543 kB/s | 32 kB 00:00 2024-09-10T09:51:16.908 INFO:teuthology.orchestra.run.smithi045.stdout:(14/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 8.4 MB/s | 11 MB 00:01 2024-09-10T09:51:16.942 INFO:teuthology.orchestra.run.smithi045.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 4.6 MB/s | 751 kB 00:00 2024-09-10T09:51:16.967 INFO:teuthology.orchestra.run.smithi045.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.7 MB/s | 164 kB 00:00 2024-09-10T09:51:16.992 INFO:teuthology.orchestra.run.smithi045.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T09:51:17.043 INFO:teuthology.orchestra.run.smithi045.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 6.6 MB/s | 506 kB 00:00 2024-09-10T09:51:17.118 INFO:teuthology.orchestra.run.smithi045.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 603 kB/s | 45 kB 00:00 2024-09-10T09:51:17.193 INFO:teuthology.orchestra.run.smithi045.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.8 MB/s | 136 kB 00:00 2024-09-10T09:51:17.268 INFO:teuthology.orchestra.run.smithi045.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T09:51:17.344 INFO:teuthology.orchestra.run.smithi045.stdout:(22/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-10T09:51:17.369 INFO:teuthology.orchestra.run.smithi045.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 3.1 MB/s | 320 kB 00:00 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi045.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.9 MB/s | 301 kB 00:00 2024-09-10T09:51:17.420 INFO:teuthology.orchestra.run.smithi045.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T09:51:17.445 INFO:teuthology.orchestra.run.smithi045.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T09:51:17.495 INFO:teuthology.orchestra.run.smithi045.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-10T09:51:17.810 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-10T09:51:17.904 INFO:teuthology.orchestra.run.smithi045.stdout:(28/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 23 MB/s | 49 MB 00:02 2024-09-10T09:51:17.921 INFO:teuthology.orchestra.run.smithi045.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 75 kB/s | 32 kB 00:00 2024-09-10T09:51:17.954 INFO:teuthology.orchestra.run.smithi045.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T09:51:18.022 INFO:teuthology.orchestra.run.smithi045.stdout:(31/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 5.2 MB/s | 3.1 MB 00:00 2024-09-10T09:51:18.130 INFO:teuthology.orchestra.run.smithi045.stdout:(32/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 17 MB/s | 3.6 MB 00:00 2024-09-10T09:51:18.155 INFO:teuthology.orchestra.run.smithi045.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 1.8 MB/s | 249 kB 00:00 2024-09-10T09:51:18.181 INFO:teuthology.orchestra.run.smithi045.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 985 kB/s | 49 kB 00:00 2024-09-10T09:51:18.206 INFO:teuthology.orchestra.run.smithi045.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 326 kB/s | 16 kB 00:00 2024-09-10T09:51:18.231 INFO:teuthology.orchestra.run.smithi045.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.6 MB/s | 285 kB 00:00 2024-09-10T09:51:18.281 INFO:teuthology.orchestra.run.smithi045.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 367 kB/s | 18 kB 00:00 2024-09-10T09:51:18.365 INFO:teuthology.orchestra.run.smithi045.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 18 MB/s | 7.4 MB 00:00 2024-09-10T09:51:18.399 INFO:teuthology.orchestra.run.smithi045.stdout:(39/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 3.9 MB/s | 766 kB 00:00 2024-09-10T09:51:18.507 INFO:teuthology.orchestra.run.smithi045.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 354 kB/s | 80 kB 00:00 2024-09-10T09:51:18.558 INFO:teuthology.orchestra.run.smithi045.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-10T09:51:18.634 INFO:teuthology.orchestra.run.smithi045.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-10T09:51:18.659 INFO:teuthology.orchestra.run.smithi045.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-10T09:51:18.684 INFO:teuthology.orchestra.run.smithi045.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T09:51:18.726 INFO:teuthology.orchestra.run.smithi045.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.2 MB 00:00 2024-09-10T09:51:18.751 INFO:teuthology.orchestra.run.smithi045.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-10T09:51:18.776 INFO:teuthology.orchestra.run.smithi045.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-09-10T09:51:18.802 INFO:teuthology.orchestra.run.smithi045.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-10T09:51:18.977 INFO:teuthology.orchestra.run.smithi045.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 611 kB/s | 107 kB 00:00 2024-09-10T09:51:19.027 INFO:teuthology.orchestra.run.smithi045.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-10T09:51:19.061 INFO:teuthology.orchestra.run.smithi045.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-10T09:51:19.103 INFO:teuthology.orchestra.run.smithi045.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-10T09:51:19.128 INFO:teuthology.orchestra.run.smithi045.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-10T09:51:19.187 INFO:teuthology.orchestra.run.smithi045.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-10T09:51:19.246 INFO:teuthology.orchestra.run.smithi045.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-09-10T09:51:19.287 INFO:teuthology.orchestra.run.smithi045.stdout:(56/156): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-09-10T09:51:19.302 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 96 kB/s | 19 kB 00:00 2024-09-10T09:51:19.321 INFO:teuthology.orchestra.run.smithi045.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-10T09:51:19.346 INFO:teuthology.orchestra.run.smithi045.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-09-10T09:51:19.472 INFO:teuthology.orchestra.run.smithi045.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-10T09:51:19.514 INFO:teuthology.orchestra.run.smithi045.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 590 kB/s | 24 kB 00:00 2024-09-10T09:51:19.555 INFO:teuthology.orchestra.run.smithi045.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-09-10T09:51:19.572 INFO:teuthology.orchestra.run.smithi045.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-10T09:51:19.656 INFO:teuthology.orchestra.run.smithi045.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T09:51:19.681 INFO:teuthology.orchestra.run.smithi045.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-10T09:51:19.707 INFO:teuthology.orchestra.run.smithi045.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-10T09:51:19.857 INFO:teuthology.orchestra.run.smithi045.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-09-10T09:51:20.073 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T09:51:20.182 INFO:teuthology.orchestra.run.smithi045.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-09-10T09:51:20.366 INFO:teuthology.orchestra.run.smithi045.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.9 MB/s | 910 kB 00:00 2024-09-10T09:51:20.433 INFO:teuthology.orchestra.run.smithi045.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-10T09:51:20.542 INFO:teuthology.orchestra.run.smithi045.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-09-10T09:51:20.633 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-10T09:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-09-10T09:51:20.784 INFO:teuthology.orchestra.run.smithi045.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T09:51:20.968 INFO:teuthology.orchestra.run.smithi045.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.8 MB/s | 4.9 MB 00:01 2024-09-10T09:51:21.026 INFO:teuthology.orchestra.run.smithi045.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-10T09:51:21.135 INFO:teuthology.orchestra.run.smithi045.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T09:51:21.218 INFO:teuthology.orchestra.run.smithi045.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-10T09:51:21.269 INFO:teuthology.orchestra.run.smithi045.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi045.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-10T09:51:21.402 INFO:teuthology.orchestra.run.smithi045.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T09:51:21.453 INFO:teuthology.orchestra.run.smithi045.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-10T09:51:21.620 INFO:teuthology.orchestra.run.smithi045.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 7.1 MB/s | 6.0 MB 00:00 2024-09-10T09:51:21.695 INFO:teuthology.orchestra.run.smithi045.stdout:(82/156): postgresql-server-13.16-1.el9.x86_64. 4.6 MB/s | 5.8 MB 00:01 2024-09-10T09:51:21.720 INFO:teuthology.orchestra.run.smithi045.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-10T09:51:21.771 INFO:teuthology.orchestra.run.smithi045.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T09:51:21.796 INFO:teuthology.orchestra.run.smithi045.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-10T09:51:21.821 INFO:teuthology.orchestra.run.smithi045.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-10T09:51:21.872 INFO:teuthology.orchestra.run.smithi045.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T09:51:21.905 INFO:teuthology.orchestra.run.smithi045.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-10T09:51:21.939 INFO:teuthology.orchestra.run.smithi045.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-10T09:51:21.964 INFO:teuthology.orchestra.run.smithi045.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-09-10T09:51:21.990 INFO:teuthology.orchestra.run.smithi045.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T09:51:22.040 INFO:teuthology.orchestra.run.smithi045.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-10T09:51:22.090 INFO:teuthology.orchestra.run.smithi045.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-10T09:51:22.149 INFO:teuthology.orchestra.run.smithi045.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T09:51:22.167 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-10T09:51:22.199 INFO:teuthology.orchestra.run.smithi045.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-09-10T09:51:22.241 INFO:teuthology.orchestra.run.smithi045.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T09:51:22.300 INFO:teuthology.orchestra.run.smithi045.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-10T09:51:22.342 INFO:teuthology.orchestra.run.smithi045.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-10T09:51:22.417 INFO:teuthology.orchestra.run.smithi045.stdout:(99/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.3 MB/s | 5.1 MB 00:00 2024-09-10T09:51:22.442 INFO:teuthology.orchestra.run.smithi045.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 954 kB/s | 96 kB 00:00 2024-09-10T09:51:22.468 INFO:teuthology.orchestra.run.smithi045.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T09:51:22.493 INFO:teuthology.orchestra.run.smithi045.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-10T09:51:22.735 INFO:teuthology.orchestra.run.smithi045.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-09-10T09:51:22.827 INFO:teuthology.orchestra.run.smithi045.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-10T09:51:22.852 INFO:teuthology.orchestra.run.smithi045.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T09:51:22.878 INFO:teuthology.orchestra.run.smithi045.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-10T09:51:22.963 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-10T09:51:22.995 INFO:teuthology.orchestra.run.smithi045.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-10T09:51:23.020 INFO:teuthology.orchestra.run.smithi045.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T09:51:23.045 INFO:teuthology.orchestra.run.smithi045.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T09:51:23.070 INFO:teuthology.orchestra.run.smithi045.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T09:51:23.096 INFO:teuthology.orchestra.run.smithi045.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-10T09:51:23.137 INFO:teuthology.orchestra.run.smithi045.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-10T09:51:23.171 INFO:teuthology.orchestra.run.smithi045.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-10T09:51:23.197 INFO:teuthology.orchestra.run.smithi045.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T09:51:23.230 INFO:teuthology.orchestra.run.smithi045.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 969 kB/s | 32 kB 00:00 2024-09-10T09:51:23.281 INFO:teuthology.orchestra.run.smithi045.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 283 kB/s | 14 kB 00:00 2024-09-10T09:51:23.306 INFO:teuthology.orchestra.run.smithi045.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-10T09:51:23.340 INFO:teuthology.orchestra.run.smithi045.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-10T09:51:23.365 INFO:teuthology.orchestra.run.smithi045.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-10T09:51:23.390 INFO:teuthology.orchestra.run.smithi045.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-10T09:51:23.424 INFO:teuthology.orchestra.run.smithi045.stdout:(121/156): protobuf-compiler-3.14.0-13.el9.x86_ 928 kB/s | 863 kB 00:00 2024-09-10T09:51:23.474 INFO:teuthology.orchestra.run.smithi045.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-10T09:51:23.500 INFO:teuthology.orchestra.run.smithi045.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T09:51:23.525 INFO:teuthology.orchestra.run.smithi045.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-10T09:51:23.550 INFO:teuthology.orchestra.run.smithi045.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-10T09:51:23.576 INFO:teuthology.orchestra.run.smithi045.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-10T09:51:23.601 INFO:teuthology.orchestra.run.smithi045.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-10T09:51:23.643 INFO:teuthology.orchestra.run.smithi045.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-10T09:51:23.668 INFO:teuthology.orchestra.run.smithi045.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T09:51:23.693 INFO:teuthology.orchestra.run.smithi045.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T09:51:23.719 INFO:teuthology.orchestra.run.smithi045.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T09:51:23.752 INFO:teuthology.orchestra.run.smithi045.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-10T09:51:23.778 INFO:teuthology.orchestra.run.smithi045.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-10T09:51:23.811 INFO:teuthology.orchestra.run.smithi045.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-09-10T09:51:23.837 INFO:teuthology.orchestra.run.smithi045.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T09:51:23.853 INFO:teuthology.orchestra.run.smithi045.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-09-10T09:51:23.879 INFO:teuthology.orchestra.run.smithi045.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-10T09:51:23.904 INFO:teuthology.orchestra.run.smithi045.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T09:51:23.929 INFO:teuthology.orchestra.run.smithi045.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T09:51:23.955 INFO:teuthology.orchestra.run.smithi045.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T09:51:23.980 INFO:teuthology.orchestra.run.smithi045.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T09:51:24.005 INFO:teuthology.orchestra.run.smithi045.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T09:51:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T09:51:24.048 INFO:teuthology.orchestra.run.smithi045.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T09:51:24.082 INFO:teuthology.orchestra.run.smithi045.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-10T09:51:24.107 INFO:teuthology.orchestra.run.smithi045.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T09:51:24.165 INFO:teuthology.orchestra.run.smithi045.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T09:51:24.407 INFO:teuthology.orchestra.run.smithi045.stdout:(148/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-10T09:51:24.491 INFO:teuthology.orchestra.run.smithi045.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 8.9 MB/s | 3.4 MB 00:00 2024-09-10T09:51:24.692 INFO:teuthology.orchestra.run.smithi045.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 8.2 MB/s | 2.3 MB 00:00 2024-09-10T09:51:24.859 INFO:teuthology.orchestra.run.smithi045.stdout:(151/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-09-10T09:51:24.884 INFO:teuthology.orchestra.run.smithi045.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 212 kB/s | 41 kB 00:00 2024-09-10T09:51:24.951 INFO:teuthology.orchestra.run.smithi045.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-10T09:51:24.993 INFO:teuthology.orchestra.run.smithi045.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 3.6 MB/s | 1.8 MB 00:00 2024-09-10T09:51:25.103 INFO:teuthology.orchestra.run.smithi045.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-10T09:51:25.103 INFO:teuthology.orchestra.run.smithi045.stdout:(156/156): NetworkManager-tui-1.49.90-1.el9.x86 1.1 MB/s | 249 kB 00:00 2024-09-10T09:51:25.103 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:51:25.103 INFO:teuthology.orchestra.run.smithi045.stdout:Total 21 MB/s | 226 MB 00:10 2024-09-10T09:51:25.863 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-10T09:51:26.346 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-09-10T09:51:26.514 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-09-10T09:51:26.514 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-09-10T09:51:28.326 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:51:28.327 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:51:28.341 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:51:28.342 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:51:28.343 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:51:28.344 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:51:28.345 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:51:28.349 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:51:28.349 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:51:28.369 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T09:51:28.416 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-09-10T09:51:28.436 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================== 2024-09-10T09:51:28.436 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-09-10T09:51:28.436 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================== 2024-09-10T09:51:28.436 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:51:28.437 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:51:28.438 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:51:28.439 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:51:28.440 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:51:28.441 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:51:28.442 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:51:28.443 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:51:28.444 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:51:28.445 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:51:28.446 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:51:28.447 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-09-10T09:51:28.448 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout:====================================================================================== 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout:Install 148 Packages 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 8 Packages 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 226 M 2024-09-10T09:51:28.449 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-09-10T09:51:28.851 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-09-10T09:51:28.852 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-09-10T09:51:29.014 INFO:teuthology.orchestra.run.smithi047.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 55 kB/s | 6.5 kB 00:00 2024-09-10T09:51:29.206 INFO:teuthology.orchestra.run.smithi047.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 4.6 MB/s | 897 kB 00:00 2024-09-10T09:51:29.314 INFO:teuthology.orchestra.run.smithi047.stdout:(3/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 13 MB/s | 5.4 MB 00:00 2024-09-10T09:51:29.340 INFO:teuthology.orchestra.run.smithi047.stdout:(4/156): ceph-immutable-object-cache-19.1.1-305 1.1 MB/s | 145 kB 00:00 2024-09-10T09:51:29.473 INFO:teuthology.orchestra.run.smithi047.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 15 MB/s | 2.4 MB 00:00 2024-09-10T09:51:29.515 INFO:teuthology.orchestra.run.smithi047.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-10T09:51:29.874 INFO:teuthology.orchestra.run.smithi047.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-10T09:51:30.374 INFO:teuthology.orchestra.run.smithi047.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 14 MB/s | 21 MB 00:01 2024-09-10T09:51:30.416 INFO:teuthology.orchestra.run.smithi047.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 609 kB/s | 25 kB 00:00 2024-09-10T09:51:30.533 INFO:teuthology.orchestra.run.smithi047.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 16 MB/s | 11 MB 00:00 2024-09-10T09:51:30.616 INFO:teuthology.orchestra.run.smithi047.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 2.7 MB/s | 228 kB 00:00 2024-09-10T09:51:30.629 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:51:30.629 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:51:30.640 INFO:teuthology.orchestra.run.smithi162.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:51:30.641 INFO:teuthology.orchestra.run.smithi162.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:51:30.642 INFO:teuthology.orchestra.run.smithi162.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:51:30.643 INFO:teuthology.orchestra.run.smithi162.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:51:30.644 INFO:teuthology.orchestra.run.smithi162.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:51:30.647 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:51:30.647 INFO:teuthology.orchestra.run.smithi162.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:51:30.710 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:51:30.734 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:51:30.735 INFO:teuthology.orchestra.run.smithi162.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:51:30.736 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:51:30.737 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:51:30.738 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:51:30.739 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:51:30.740 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:51:30.741 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:51:30.742 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:51:30.743 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:51:30.744 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:51:30.745 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout:====================================================================================== 2024-09-10T09:51:30.746 INFO:teuthology.orchestra.run.smithi162.stdout:Install 148 Packages 2024-09-10T09:51:30.747 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 8 Packages 2024-09-10T09:51:30.747 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:51:30.747 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 226 M 2024-09-10T09:51:30.747 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T09:51:30.800 INFO:teuthology.orchestra.run.smithi047.stdout:(12/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 13 MB/s | 17 MB 00:01 2024-09-10T09:51:30.825 INFO:teuthology.orchestra.run.smithi047.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 129 kB/s | 27 kB 00:00 2024-09-10T09:51:30.850 INFO:teuthology.orchestra.run.smithi047.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 634 kB/s | 32 kB 00:00 2024-09-10T09:51:30.893 INFO:teuthology.orchestra.run.smithi047.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 751 kB 00:00 2024-09-10T09:51:30.918 INFO:teuthology.orchestra.run.smithi047.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.4 MB/s | 164 kB 00:00 2024-09-10T09:51:30.943 INFO:teuthology.orchestra.run.smithi047.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T09:51:30.977 INFO:teuthology.orchestra.run.smithi047.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 8.5 MB/s | 506 kB 00:00 2024-09-10T09:51:31.035 INFO:teuthology.orchestra.run.smithi047.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 774 kB/s | 45 kB 00:00 2024-09-10T09:51:31.161 INFO:teuthology.orchestra.run.smithi047.stdout:(20/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 25 MB/s | 5.3 MB 00:00 2024-09-10T09:51:31.186 INFO:teuthology.orchestra.run.smithi047.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 905 kB/s | 136 kB 00:00 2024-09-10T09:51:31.211 INFO:teuthology.orchestra.run.smithi047.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.2 MB/s | 161 kB 00:00 2024-09-10T09:51:31.237 INFO:teuthology.orchestra.run.smithi047.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 6.2 MB/s | 320 kB 00:00 2024-09-10T09:51:31.254 INFO:teuthology.orchestra.run.smithi162.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 36 kB/s | 6.5 kB 00:00 2024-09-10T09:51:31.262 INFO:teuthology.orchestra.run.smithi047.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.8 MB/s | 301 kB 00:00 2024-09-10T09:51:31.287 INFO:teuthology.orchestra.run.smithi047.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T09:51:31.312 INFO:teuthology.orchestra.run.smithi047.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T09:51:31.371 INFO:teuthology.orchestra.run.smithi047.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-10T09:51:31.463 INFO:teuthology.orchestra.run.smithi047.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 348 kB/s | 32 kB 00:00 2024-09-10T09:51:31.522 INFO:teuthology.orchestra.run.smithi047.stdout:(29/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-10T09:51:31.555 INFO:teuthology.orchestra.run.smithi047.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.6 MB/s | 146 kB 00:00 2024-09-10T09:51:31.729 INFO:teuthology.orchestra.run.smithi162.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.8 MB/s | 897 kB 00:00 2024-09-10T09:51:31.813 INFO:teuthology.orchestra.run.smithi162.stdout:(3/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 7.3 MB/s | 5.4 MB 00:00 2024-09-10T09:51:31.838 INFO:teuthology.orchestra.run.smithi162.stdout:(4/156): ceph-immutable-object-cache-19.1.1-305 1.3 MB/s | 145 kB 00:00 2024-09-10T09:51:31.914 INFO:teuthology.orchestra.run.smithi047.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 9.1 MB/s | 3.6 MB 00:00 2024-09-10T09:51:32.022 INFO:teuthology.orchestra.run.smithi047.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.3 MB/s | 249 kB 00:00 2024-09-10T09:51:32.114 INFO:teuthology.orchestra.run.smithi047.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 538 kB/s | 49 kB 00:00 2024-09-10T09:51:32.121 INFO:teuthology.orchestra.run.smithi162.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 7.7 MB/s | 2.4 MB 00:00 2024-09-10T09:51:32.206 INFO:teuthology.orchestra.run.smithi047.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi047.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 151 kB/s | 16 kB 00:00 2024-09-10T09:51:32.299 INFO:teuthology.orchestra.run.smithi047.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 3.0 MB/s | 285 kB 00:00 2024-09-10T09:51:32.347 INFO:teuthology.orchestra.run.smithi162.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 3.0 MB/s | 1.5 MB 00:00 2024-09-10T09:51:32.390 INFO:teuthology.orchestra.run.smithi047.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 4.5 MB/s | 766 kB 00:00 2024-09-10T09:51:32.407 INFO:teuthology.orchestra.run.smithi047.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 168 kB/s | 18 kB 00:00 2024-09-10T09:51:32.516 INFO:teuthology.orchestra.run.smithi047.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 638 kB/s | 80 kB 00:00 2024-09-10T09:51:32.575 INFO:teuthology.orchestra.run.smithi047.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T09:51:32.600 INFO:teuthology.orchestra.run.smithi047.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-10T09:51:32.642 INFO:teuthology.orchestra.run.smithi047.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-10T09:51:32.692 INFO:teuthology.orchestra.run.smithi047.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T09:51:32.722 INFO:teuthology.orchestra.run.smithi162.stdout:(7/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 21 MB 00:01 2024-09-10T09:51:32.736 INFO:teuthology.orchestra.run.smithi047.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-10T09:51:32.785 INFO:teuthology.orchestra.run.smithi047.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T09:51:32.814 INFO:teuthology.orchestra.run.smithi162.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 6.8 MB/s | 4.7 MB 00:00 2024-09-10T09:51:32.835 INFO:teuthology.orchestra.run.smithi047.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-10T09:51:32.860 INFO:teuthology.orchestra.run.smithi047.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-10T09:51:32.881 INFO:teuthology.orchestra.run.smithi162.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 378 kB/s | 25 kB 00:00 2024-09-10T09:51:33.239 INFO:teuthology.orchestra.run.smithi162.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 21 MB/s | 11 MB 00:00 2024-09-10T09:51:33.289 INFO:teuthology.orchestra.run.smithi162.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 4.5 MB/s | 228 kB 00:00 2024-09-10T09:51:33.348 INFO:teuthology.orchestra.run.smithi162.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 464 kB/s | 27 kB 00:00 2024-09-10T09:51:33.402 INFO:teuthology.orchestra.run.smithi047.stdout:(48/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 16 MB/s | 49 MB 00:02 2024-09-10T09:51:33.407 INFO:teuthology.orchestra.run.smithi162.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 544 kB/s | 32 kB 00:00 2024-09-10T09:51:33.499 INFO:teuthology.orchestra.run.smithi162.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-10T09:51:33.557 INFO:teuthology.orchestra.run.smithi162.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.7 MB/s | 164 kB 00:00 2024-09-10T09:51:33.616 INFO:teuthology.orchestra.run.smithi162.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 2.1 MB/s | 127 kB 00:00 2024-09-10T09:51:33.733 INFO:teuthology.orchestra.run.smithi162.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 4.2 MB/s | 506 kB 00:00 2024-09-10T09:51:33.900 INFO:teuthology.orchestra.run.smithi162.stdout:(18/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 17 MB 00:01 2024-09-10T09:51:33.950 INFO:teuthology.orchestra.run.smithi162.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 905 kB/s | 45 kB 00:00 2024-09-10T09:51:34.026 INFO:teuthology.orchestra.run.smithi162.stdout:(20/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 18 MB/s | 5.3 MB 00:00 2024-09-10T09:51:34.042 INFO:teuthology.orchestra.run.smithi162.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 1.4 MB/s | 136 kB 00:00 2024-09-10T09:51:34.076 INFO:teuthology.orchestra.run.smithi162.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.1 MB/s | 161 kB 00:00 2024-09-10T09:51:34.085 INFO:teuthology.orchestra.run.smithi047.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 158 kB/s | 107 kB 00:00 2024-09-10T09:51:34.101 INFO:teuthology.orchestra.run.smithi162.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 5.3 MB/s | 320 kB 00:00 2024-09-10T09:51:34.135 INFO:teuthology.orchestra.run.smithi162.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T09:51:34.160 INFO:teuthology.orchestra.run.smithi162.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.7 MB/s | 99 kB 00:00 2024-09-10T09:51:34.169 INFO:teuthology.orchestra.run.smithi047.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 516 kB/s | 689 kB 00:01 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi162.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T09:51:34.203 INFO:teuthology.orchestra.run.smithi047.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 702 kB/s | 943 kB 00:01 2024-09-10T09:51:34.245 INFO:teuthology.orchestra.run.smithi162.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.8 MB/s | 170 kB 00:00 2024-09-10T09:51:34.312 INFO:teuthology.orchestra.run.smithi162.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 477 kB/s | 32 kB 00:00 2024-09-10T09:51:34.362 INFO:teuthology.orchestra.run.smithi162.stdout:(29/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 15 MB/s | 3.1 MB 00:00 2024-09-10T09:51:34.387 INFO:teuthology.orchestra.run.smithi162.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.9 MB/s | 146 kB 00:00 2024-09-10T09:51:34.654 INFO:teuthology.orchestra.run.smithi162.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 12 MB/s | 3.6 MB 00:00 2024-09-10T09:51:34.670 INFO:teuthology.orchestra.run.smithi047.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 37 kB/s | 22 kB 00:00 2024-09-10T09:51:34.746 INFO:teuthology.orchestra.run.smithi162.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 21 MB/s | 7.4 MB 00:00 2024-09-10T09:51:34.771 INFO:teuthology.orchestra.run.smithi162.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.1 MB/s | 249 kB 00:00 2024-09-10T09:51:34.788 INFO:teuthology.orchestra.run.smithi162.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 1.2 MB/s | 49 kB 00:00 2024-09-10T09:51:34.812 INFO:teuthology.orchestra.run.smithi047.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-09-10T09:51:34.813 INFO:teuthology.orchestra.run.smithi162.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 391 kB/s | 16 kB 00:00 2024-09-10T09:51:34.847 INFO:teuthology.orchestra.run.smithi162.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.8 MB/s | 285 kB 00:00 2024-09-10T09:51:34.881 INFO:teuthology.orchestra.run.smithi162.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 11 MB/s | 766 kB 00:00 2024-09-10T09:51:34.898 INFO:teuthology.orchestra.run.smithi162.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 364 kB/s | 18 kB 00:00 2024-09-10T09:51:34.954 INFO:teuthology.orchestra.run.smithi047.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 400 kB/s | 313 kB 00:00 2024-09-10T09:51:34.982 INFO:teuthology.orchestra.run.smithi162.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 797 kB/s | 80 kB 00:00 2024-09-10T09:51:35.040 INFO:teuthology.orchestra.run.smithi162.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T09:51:35.065 INFO:teuthology.orchestra.run.smithi162.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-09-10T09:51:35.091 INFO:teuthology.orchestra.run.smithi162.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-09-10T09:51:35.133 INFO:teuthology.orchestra.run.smithi162.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T09:51:35.329 INFO:teuthology.orchestra.run.smithi047.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-09-10T09:51:35.437 INFO:teuthology.orchestra.run.smithi047.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-10T09:51:35.462 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-09-10T09:51:35.539 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:51:35.578 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:51:35.632 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:51:35.633 INFO:teuthology.orchestra.run.smithi162.stdout:(44/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 18 MB/s | 49 MB 00:02 2024-09-10T09:51:35.658 INFO:teuthology.orchestra.run.smithi162.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 257 kB/s | 135 kB 00:00 2024-09-10T09:51:35.673 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:51:35.692 INFO:teuthology.orchestra.run.smithi162.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 2.0 MB/s | 1.2 MB 00:00 2024-09-10T09:51:35.698 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:51:35.734 INFO:teuthology.orchestra.run.smithi162.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-10T09:51:35.738 INFO:teuthology.orchestra.run.smithi047.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-10T09:51:35.754 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:51:35.776 INFO:teuthology.orchestra.run.smithi162.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 879 kB/s | 125 kB 00:00 2024-09-10T09:51:35.884 INFO:teuthology.orchestra.run.smithi162.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-09-10T09:51:35.910 INFO:teuthology.orchestra.run.smithi162.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 800 kB/s | 107 kB 00:00 2024-09-10T09:51:35.935 INFO:teuthology.orchestra.run.smithi162.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-10T09:51:35.977 INFO:teuthology.orchestra.run.smithi162.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-10T09:51:36.002 INFO:teuthology.orchestra.run.smithi162.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-09-10T09:51:36.028 INFO:teuthology.orchestra.run.smithi162.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T09:51:36.030 INFO:teuthology.orchestra.run.smithi047.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 76 kB/s | 45 kB 00:00 2024-09-10T09:51:36.061 INFO:teuthology.orchestra.run.smithi162.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 309 kB/s | 18 kB 00:00 2024-09-10T09:51:36.080 INFO:teuthology.orchestra.run.smithi047.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 214 kB/s | 160 kB 00:00 2024-09-10T09:51:36.087 INFO:teuthology.orchestra.run.smithi162.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 189 kB/s | 11 kB 00:00 2024-09-10T09:51:36.112 INFO:teuthology.orchestra.run.smithi162.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T09:51:36.137 INFO:teuthology.orchestra.run.smithi162.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-10T09:51:36.169 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:51:36.171 INFO:teuthology.orchestra.run.smithi162.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-10T09:51:36.204 INFO:teuthology.orchestra.run.smithi162.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-10T09:51:36.230 INFO:teuthology.orchestra.run.smithi162.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-10T09:51:36.255 INFO:teuthology.orchestra.run.smithi162.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-10T09:51:36.275 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:51:36.280 INFO:teuthology.orchestra.run.smithi162.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T09:51:36.306 INFO:teuthology.orchestra.run.smithi162.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T09:51:36.331 INFO:teuthology.orchestra.run.smithi162.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-10T09:51:36.381 INFO:teuthology.orchestra.run.smithi162.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-09-10T09:51:36.383 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:51:36.409 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:51:36.448 INFO:teuthology.orchestra.run.smithi162.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-09-10T09:51:36.502 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:51:36.524 INFO:teuthology.orchestra.run.smithi162.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 23 MB/s | 4.9 MB 00:00 2024-09-10T09:51:36.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:51:36.590 INFO:teuthology.orchestra.run.smithi162.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-10T09:51:36.597 INFO:teuthology.orchestra.run.smithi047.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 772 kB/s | 662 kB 00:00 2024-09-10T09:51:36.616 INFO:teuthology.orchestra.run.smithi162.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-10T09:51:36.649 INFO:teuthology.orchestra.run.smithi162.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-10T09:51:36.655 INFO:teuthology.orchestra.run.smithi047.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 43 kB/s | 24 kB 00:00 2024-09-10T09:51:36.683 INFO:teuthology.orchestra.run.smithi162.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-10T09:51:36.708 INFO:teuthology.orchestra.run.smithi162.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T09:51:36.715 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:51:36.750 INFO:teuthology.orchestra.run.smithi162.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T09:51:36.761 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:51:36.789 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:51:36.826 INFO:teuthology.orchestra.run.smithi162.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-10T09:51:36.851 INFO:teuthology.orchestra.run.smithi162.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T09:51:36.876 INFO:teuthology.orchestra.run.smithi162.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-10T09:51:36.898 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:51:36.901 INFO:teuthology.orchestra.run.smithi162.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-10T09:51:36.923 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:51:36.927 INFO:teuthology.orchestra.run.smithi162.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-10T09:51:36.957 INFO:teuthology.orchestra.run.smithi162.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.0 MB/s | 172 kB 00:00 2024-09-10T09:51:36.964 INFO:teuthology.orchestra.run.smithi047.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 623 kB/s | 581 kB 00:00 2024-09-10T09:51:36.974 INFO:teuthology.orchestra.run.smithi162.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 735 kB/s | 35 kB 00:00 2024-09-10T09:51:36.978 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:51:37.008 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:51:37.032 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:51:37.050 INFO:teuthology.orchestra.run.smithi162.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-10T09:51:37.075 INFO:teuthology.orchestra.run.smithi162.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-10T09:51:37.100 INFO:teuthology.orchestra.run.smithi162.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T09:51:37.126 INFO:teuthology.orchestra.run.smithi162.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T09:51:37.151 INFO:teuthology.orchestra.run.smithi162.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-10T09:51:37.176 INFO:teuthology.orchestra.run.smithi162.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-10T09:51:37.181 INFO:teuthology.orchestra.run.smithi047.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-10T09:51:37.193 INFO:teuthology.orchestra.run.smithi162.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-09-10T09:51:37.218 INFO:teuthology.orchestra.run.smithi162.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-09-10T09:51:37.244 INFO:teuthology.orchestra.run.smithi162.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T09:51:37.269 INFO:teuthology.orchestra.run.smithi162.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T09:51:37.353 INFO:teuthology.orchestra.run.smithi162.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-10T09:51:37.378 INFO:teuthology.orchestra.run.smithi162.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-10T09:51:37.403 INFO:teuthology.orchestra.run.smithi162.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-10T09:51:37.429 INFO:teuthology.orchestra.run.smithi162.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T09:51:37.454 INFO:teuthology.orchestra.run.smithi162.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-10T09:51:37.479 INFO:teuthology.orchestra.run.smithi162.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-09-10T09:51:37.505 INFO:teuthology.orchestra.run.smithi162.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T09:51:37.506 INFO:teuthology.orchestra.run.smithi047.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 344 kB/s | 292 kB 00:00 2024-09-10T09:51:37.510 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:51:37.530 INFO:teuthology.orchestra.run.smithi162.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T09:51:37.555 INFO:teuthology.orchestra.run.smithi162.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T09:51:37.580 INFO:teuthology.orchestra.run.smithi162.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T09:51:37.606 INFO:teuthology.orchestra.run.smithi162.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-10T09:51:37.677 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:51:37.748 INFO:teuthology.orchestra.run.smithi047.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-10T09:51:37.782 INFO:teuthology.orchestra.run.smithi047.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 231 kB/s | 188 kB 00:00 2024-09-10T09:51:37.829 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:51:37.831 INFO:teuthology.orchestra.run.smithi162.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-10T09:51:37.890 INFO:teuthology.orchestra.run.smithi162.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-10T09:51:37.923 INFO:teuthology.orchestra.run.smithi162.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T09:51:37.925 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:51:37.949 INFO:teuthology.orchestra.run.smithi162.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 789 kB/s | 19 kB 00:00 2024-09-10T09:51:38.022 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:51:38.049 INFO:teuthology.orchestra.run.smithi162.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-10T09:51:38.091 INFO:teuthology.orchestra.run.smithi162.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 946 kB/s | 26 kB 00:00 2024-09-10T09:51:38.116 INFO:teuthology.orchestra.run.smithi162.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T09:51:38.133 INFO:teuthology.orchestra.run.smithi162.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 4.0 MB/s | 67 kB 00:00 2024-09-10T09:51:38.158 INFO:teuthology.orchestra.run.smithi162.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.1 MB/s | 151 kB 00:00 2024-09-10T09:51:38.200 INFO:teuthology.orchestra.run.smithi162.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-10T09:51:38.234 INFO:teuthology.orchestra.run.smithi162.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-10T09:51:38.248 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:51:38.259 INFO:teuthology.orchestra.run.smithi162.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T09:51:38.284 INFO:teuthology.orchestra.run.smithi162.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T09:51:38.310 INFO:teuthology.orchestra.run.smithi162.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-10T09:51:38.335 INFO:teuthology.orchestra.run.smithi162.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-10T09:51:38.356 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:51:38.368 INFO:teuthology.orchestra.run.smithi162.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-10T09:51:38.394 INFO:teuthology.orchestra.run.smithi162.stdout:(119/156): lua-devel-5.4.4-4.el9.x86_64.rpm 27 kB/s | 22 kB 00:00 2024-09-10T09:51:38.427 INFO:teuthology.orchestra.run.smithi162.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T09:51:38.451 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:51:38.461 INFO:teuthology.orchestra.run.smithi162.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-10T09:51:38.486 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:51:38.486 INFO:teuthology.orchestra.run.smithi162.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T09:51:38.503 INFO:teuthology.orchestra.run.smithi162.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 641 kB/s | 11 kB 00:00 2024-09-10T09:51:38.520 INFO:teuthology.orchestra.run.smithi162.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T09:51:38.521 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:51:38.546 INFO:teuthology.orchestra.run.smithi162.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 916 kB/s | 23 kB 00:00 2024-09-10T09:51:38.571 INFO:teuthology.orchestra.run.smithi162.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-10T09:51:38.597 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:51:38.622 INFO:teuthology.orchestra.run.smithi162.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-10T09:51:38.640 INFO:teuthology.orchestra.run.smithi047.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-09-10T09:51:38.647 INFO:teuthology.orchestra.run.smithi162.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-09-10T09:51:38.647 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:51:38.672 INFO:teuthology.orchestra.run.smithi162.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T09:51:38.687 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:51:38.706 INFO:teuthology.orchestra.run.smithi162.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T09:51:38.731 INFO:teuthology.orchestra.run.smithi162.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T09:51:38.731 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:51:38.763 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:51:38.774 INFO:teuthology.orchestra.run.smithi047.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-10T09:51:38.791 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:51:38.834 INFO:teuthology.orchestra.run.smithi162.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T09:51:38.863 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:51:38.874 INFO:teuthology.orchestra.run.smithi162.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 421 kB/s | 16 kB 00:00 2024-09-10T09:51:38.874 INFO:teuthology.orchestra.run.smithi047.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-09-10T09:51:38.915 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:51:38.940 INFO:teuthology.orchestra.run.smithi162.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-09-10T09:51:38.944 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:51:38.966 INFO:teuthology.orchestra.run.smithi162.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 980 kB/s | 90 kB 00:00 2024-09-10T09:51:38.979 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:51:38.991 INFO:teuthology.orchestra.run.smithi162.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-10T09:51:39.024 INFO:teuthology.orchestra.run.smithi162.stdout:(137/156): protobuf-compiler-3.14.0-13.el9.x86_ 609 kB/s | 863 kB 00:01 2024-09-10T09:51:39.050 INFO:teuthology.orchestra.run.smithi162.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-09-10T09:51:39.059 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:51:39.090 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:51:39.092 INFO:teuthology.orchestra.run.smithi162.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-10T09:51:39.117 INFO:teuthology.orchestra.run.smithi162.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-10T09:51:39.142 INFO:teuthology.orchestra.run.smithi162.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-09-10T09:51:39.145 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:51:39.159 INFO:teuthology.orchestra.run.smithi162.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-10T09:51:39.171 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:51:39.184 INFO:teuthology.orchestra.run.smithi162.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-10T09:51:39.210 INFO:teuthology.orchestra.run.smithi162.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-10T09:51:39.235 INFO:teuthology.orchestra.run.smithi162.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-10T09:51:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:51:39.260 INFO:teuthology.orchestra.run.smithi162.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T09:51:39.286 INFO:teuthology.orchestra.run.smithi162.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T09:51:39.286 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:51:39.336 INFO:teuthology.orchestra.run.smithi162.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T09:51:39.365 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:51:39.508 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:51:39.536 INFO:teuthology.orchestra.run.smithi162.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-10T09:51:39.557 INFO:teuthology.orchestra.run.smithi047.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 175 kB/s | 137 kB 00:00 2024-09-10T09:51:39.565 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:51:39.587 INFO:teuthology.orchestra.run.smithi162.stdout:(150/156): librados2-19.1.1-305.g1cf819b7.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-10T09:51:39.633 INFO:teuthology.orchestra.run.smithi047.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 917 kB/s | 910 kB 00:00 2024-09-10T09:51:39.645 INFO:teuthology.orchestra.run.smithi162.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 700 kB/s | 41 kB 00:00 2024-09-10T09:51:39.746 INFO:teuthology.orchestra.run.smithi162.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.4 MB/s | 249 kB 00:00 2024-09-10T09:51:39.988 INFO:teuthology.orchestra.run.smithi162.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 4.0 MB/s | 1.8 MB 00:00 2024-09-10T09:51:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-09-10T09:51:40.038 INFO:teuthology.orchestra.run.smithi162.stdout:(154/156): NetworkManager-1.49.90-1.el9.x86_64. 3.3 MB/s | 2.3 MB 00:00 2024-09-10T09:51:40.088 INFO:teuthology.orchestra.run.smithi162.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 1.4 MB 00:00 2024-09-10T09:51:40.118 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:51:40.211 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:51:40.247 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:51:40.291 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:51:40.308 INFO:teuthology.orchestra.run.smithi047.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 130 kB/s | 87 kB 00:00 2024-09-10T09:51:40.328 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:51:40.370 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:51:40.548 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:51:40.598 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:51:40.625 INFO:teuthology.orchestra.run.smithi047.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 968 kB/s | 1.0 MB 00:01 2024-09-10T09:51:40.655 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:51:40.680 INFO:teuthology.orchestra.run.smithi162.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-09-10T09:51:40.687 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:51:40.687 INFO:teuthology.orchestra.run.smithi162.stdout:Total 23 MB/s | 226 MB 00:09 2024-09-10T09:51:40.730 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:51:40.765 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:51:40.802 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:51:40.858 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:51:40.893 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:51:40.937 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:51:40.967 INFO:teuthology.orchestra.run.smithi047.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 93 kB/s | 61 kB 00:00 2024-09-10T09:51:40.980 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:51:41.131 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:51:41.177 INFO:teuthology.orchestra.run.smithi047.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-10T09:51:41.339 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:51:41.381 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:51:41.411 INFO:teuthology.orchestra.run.smithi047.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 312 kB/s | 245 kB 00:00 2024-09-10T09:51:41.507 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:51:41.545 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:51:41.579 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:51:41.612 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:51:41.637 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:51:41.735 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:51:41.777 INFO:teuthology.orchestra.run.smithi047.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-09-10T09:51:41.803 INFO:teuthology.orchestra.run.smithi047.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 297 kB/s | 248 kB 00:00 2024-09-10T09:51:41.869 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:51:41.877 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T09:51:41.903 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:51:41.928 INFO:teuthology.orchestra.run.smithi047.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-09-10T09:51:41.931 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:51:41.969 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:51:42.005 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:51:42.028 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T09:51:42.028 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T09:51:42.045 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:51:42.075 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:51:42.101 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:51:42.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:51:42.176 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:51:42.210 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:51:42.301 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:51:42.350 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:51:42.370 INFO:teuthology.orchestra.run.smithi047.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 61 kB/s | 35 kB 00:00 2024-09-10T09:51:42.380 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:51:42.488 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:51:42.550 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:51:42.578 INFO:teuthology.orchestra.run.smithi047.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 215 kB/s | 172 kB 00:00 2024-09-10T09:51:43.054 INFO:teuthology.orchestra.run.smithi047.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-10T09:51:43.137 INFO:teuthology.orchestra.run.smithi047.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 327 kB/s | 250 kB 00:00 2024-09-10T09:51:43.312 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:51:43.354 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:51:43.388 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:51:43.396 INFO:teuthology.orchestra.run.smithi047.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 272 kB/s | 222 kB 00:00 2024-09-10T09:51:43.496 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:51:43.548 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:51:43.593 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:51:43.683 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:51:43.696 INFO:teuthology.orchestra.run.smithi047.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 120 kB/s | 77 kB 00:00 2024-09-10T09:51:43.721 INFO:teuthology.orchestra.run.smithi047.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 72 kB/s | 42 kB 00:00 2024-09-10T09:51:43.728 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:51:43.840 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T09:51:43.840 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T09:51:43.919 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:51:43.959 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:51:44.039 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:51:44.171 INFO:teuthology.orchestra.run.smithi047.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 347 kB/s | 269 kB 00:00 2024-09-10T09:51:44.405 INFO:teuthology.orchestra.run.smithi047.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 224 kB/s | 159 kB 00:00 2024-09-10T09:51:44.505 INFO:teuthology.orchestra.run.smithi047.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 356 kB/s | 279 kB 00:00 2024-09-10T09:51:44.731 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:51:44.814 INFO:teuthology.orchestra.run.smithi047.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-09-10T09:51:44.854 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:51:45.064 INFO:teuthology.orchestra.run.smithi047.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 82 kB/s | 54 kB 00:00 2024-09-10T09:51:45.381 INFO:teuthology.orchestra.run.smithi047.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 74 kB/s | 42 kB 00:00 2024-09-10T09:51:45.690 INFO:teuthology.orchestra.run.smithi047.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-09-10T09:51:45.932 INFO:teuthology.orchestra.run.smithi047.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-10T09:51:46.172 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:51:46.223 INFO:teuthology.orchestra.run.smithi047.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 262 kB/s | 221 kB 00:00 2024-09-10T09:51:46.247 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:51:46.283 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:51:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:51:46.390 INFO:teuthology.orchestra.run.smithi047.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 51 kB/s | 36 kB 00:00 2024-09-10T09:51:46.507 INFO:teuthology.orchestra.run.smithi047.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-10T09:51:46.644 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:51:46.687 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:51:46.787 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:51:46.901 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:51:46.938 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:51:46.999 INFO:teuthology.orchestra.run.smithi047.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-10T09:51:47.025 INFO:teuthology.orchestra.run.smithi047.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 380 kB/s | 304 kB 00:00 2024-09-10T09:51:47.166 INFO:teuthology.orchestra.run.smithi047.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 145 kB/s | 96 kB 00:00 2024-09-10T09:51:47.258 INFO:teuthology.orchestra.run.smithi047.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-09-10T09:51:47.444 INFO:teuthology.orchestra.run.smithi047.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-09-10T09:51:47.446 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:51:47.474 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:51:47.492 INFO:teuthology.orchestra.run.smithi047.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T09:51:47.526 INFO:teuthology.orchestra.run.smithi047.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-10T09:51:47.545 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:51:47.551 INFO:teuthology.orchestra.run.smithi047.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-10T09:51:47.576 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:51:47.660 INFO:teuthology.orchestra.run.smithi047.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T09:51:47.685 INFO:teuthology.orchestra.run.smithi047.stdout:(108/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-09-10T09:51:47.710 INFO:teuthology.orchestra.run.smithi047.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-10T09:51:47.744 INFO:teuthology.orchestra.run.smithi047.stdout:(110/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 86 kB/s | 64 kB 00:00 2024-09-10T09:51:47.778 INFO:teuthology.orchestra.run.smithi047.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-10T09:51:47.811 INFO:teuthology.orchestra.run.smithi047.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 386 kB/s | 49 kB 00:00 2024-09-10T09:51:47.853 INFO:teuthology.orchestra.run.smithi047.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-10T09:51:47.878 INFO:teuthology.orchestra.run.smithi047.stdout:(114/156): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-10T09:51:47.904 INFO:teuthology.orchestra.run.smithi047.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-09-10T09:51:47.937 INFO:teuthology.orchestra.run.smithi047.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-09-10T09:51:47.963 INFO:teuthology.orchestra.run.smithi047.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 241 kB/s | 14 kB 00:00 2024-09-10T09:51:47.988 INFO:teuthology.orchestra.run.smithi047.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-09-10T09:51:48.005 INFO:teuthology.orchestra.run.smithi047.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-10T09:51:48.030 INFO:teuthology.orchestra.run.smithi047.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T09:51:48.055 INFO:teuthology.orchestra.run.smithi047.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-10T09:51:48.081 INFO:teuthology.orchestra.run.smithi047.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-10T09:51:48.097 INFO:teuthology.orchestra.run.smithi047.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 643 kB/s | 11 kB 00:00 2024-09-10T09:51:48.114 INFO:teuthology.orchestra.run.smithi047.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T09:51:48.140 INFO:teuthology.orchestra.run.smithi047.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-09-10T09:51:48.157 INFO:teuthology.orchestra.run.smithi047.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 553 kB/s | 23 kB 00:00 2024-09-10T09:51:48.182 INFO:teuthology.orchestra.run.smithi047.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 463 kB/s | 19 kB 00:00 2024-09-10T09:51:48.207 INFO:teuthology.orchestra.run.smithi047.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T09:51:48.233 INFO:teuthology.orchestra.run.smithi047.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T09:51:48.275 INFO:teuthology.orchestra.run.smithi047.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-09-10T09:51:48.300 INFO:teuthology.orchestra.run.smithi047.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T09:51:48.342 INFO:teuthology.orchestra.run.smithi047.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-09-10T09:51:48.367 INFO:teuthology.orchestra.run.smithi047.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-10T09:51:48.401 INFO:teuthology.orchestra.run.smithi047.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-10T09:51:48.435 INFO:teuthology.orchestra.run.smithi047.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 840 kB/s | 28 kB 00:00 2024-09-10T09:51:48.460 INFO:teuthology.orchestra.run.smithi047.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2024-09-10T09:51:48.485 INFO:teuthology.orchestra.run.smithi047.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-10T09:51:48.510 INFO:teuthology.orchestra.run.smithi047.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T09:51:48.536 INFO:teuthology.orchestra.run.smithi047.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T09:51:48.561 INFO:teuthology.orchestra.run.smithi047.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-10T09:51:48.586 INFO:teuthology.orchestra.run.smithi047.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T09:51:48.603 INFO:teuthology.orchestra.run.smithi047.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 5.3 MB/s | 90 kB 00:00 2024-09-10T09:51:48.629 INFO:teuthology.orchestra.run.smithi047.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-10T09:51:48.724 INFO:teuthology.orchestra.run.smithi047.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 209 kB/s | 20 kB 00:00 2024-09-10T09:51:48.829 INFO:teuthology.orchestra.run.smithi047.stdout:(145/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-09-10T09:51:48.854 INFO:teuthology.orchestra.run.smithi047.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-10T09:51:48.880 INFO:teuthology.orchestra.run.smithi047.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 1.5 MB/s | 427 kB 00:00 2024-09-10T09:51:48.922 INFO:teuthology.orchestra.run.smithi047.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T09:51:49.122 INFO:teuthology.orchestra.run.smithi047.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-10T09:51:49.181 INFO:teuthology.orchestra.run.smithi047.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T09:51:49.247 INFO:teuthology.orchestra.run.smithi047.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 614 kB/s | 41 kB 00:00 2024-09-10T09:51:49.323 INFO:teuthology.orchestra.run.smithi047.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-10T09:51:49.481 INFO:teuthology.orchestra.run.smithi047.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 4.1 MB/s | 2.3 MB 00:00 2024-09-10T09:51:49.531 INFO:teuthology.orchestra.run.smithi047.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 4.4 MB/s | 1.8 MB 00:00 2024-09-10T09:51:49.565 INFO:teuthology.orchestra.run.smithi047.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-10T09:51:49.960 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:51:49.989 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:51:50.165 INFO:teuthology.orchestra.run.smithi047.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-09-10T09:51:50.172 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:51:50.172 INFO:teuthology.orchestra.run.smithi047.stdout:Total 10 MB/s | 226 MB 00:21 2024-09-10T09:51:50.260 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T09:51:50.358 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:51:50.396 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:51:50.566 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:51:50.594 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:51:50.601 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:51:50.634 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:51:50.659 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:51:50.676 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:51:50.676 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:51:50.677 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:51:50.677 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:51:50.703 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:51:50.711 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:51:51.036 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:51:51.156 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:51:51.298 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:51:51.337 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:51:51.427 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:51:51.436 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-09-10T09:51:51.458 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:51:51.605 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:51:51.606 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-09-10T09:51:51.606 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-09-10T09:51:51.654 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:51:51.683 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:51:51.769 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:51:51.792 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:51:51.854 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:51:51.877 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:51:51.898 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:51:52.053 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:51:52.187 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:51:52.309 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:51:52.422 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:51:52.485 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:51:52.715 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:51:52.823 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:51:52.920 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:51:52.950 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:51:52.984 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:51:53.057 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:51:53.109 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:51:53.141 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:51:53.185 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:51:53.218 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:51:53.246 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:51:53.316 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:51:53.368 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:51:53.399 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:51:53.426 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:51:53.502 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:51:53.505 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-09-10T09:51:53.505 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-09-10T09:51:53.537 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:51:53.580 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:51:53.609 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:51:53.688 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:51:53.732 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:51:53.808 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:51:53.939 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:51:53.997 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:51:54.515 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:51:54.589 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:51:54.622 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:51:54.665 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:51:54.704 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:51:54.744 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:51:54.891 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:51:54.948 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:51:55.004 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:51:55.076 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:51:55.107 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:51:55.143 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:51:55.190 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:51:55.235 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:51:55.287 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:51:55.328 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:51:55.466 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:51:55.513 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:51:55.555 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:51:55.659 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:51:55.798 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:51:55.837 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:51:55.863 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:51:55.880 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:51:55.954 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:51:56.087 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:51:56.126 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:51:56.154 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:51:56.205 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:51:56.244 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:51:56.275 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:51:56.305 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:51:56.340 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:51:56.373 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:51:56.416 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:51:56.450 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:51:56.537 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:51:56.581 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:51:56.612 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:51:56.713 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:51:56.770 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:51:57.491 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:51:57.535 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:51:57.568 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:51:57.668 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:51:57.724 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:51:57.769 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:51:57.857 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:51:57.888 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:51:58.073 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:51:58.103 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:51:58.180 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:51:58.827 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:51:58.944 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:52:00.158 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:52:00.215 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-09-10T09:52:00.233 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:52:00.261 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:52:00.294 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:52:00.328 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:52:00.367 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:52:00.420 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:52:00.454 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:52:00.479 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:52:00.522 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:52:00.605 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:52:00.649 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:52:00.740 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:52:00.770 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:52:00.869 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:52:00.980 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:52:01.017 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:52:01.104 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:52:01.130 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:52:01.215 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:52:01.243 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:52:01.399 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:52:01.451 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:52:01.500 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:52:01.527 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:52:01.598 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:52:01.620 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:52:01.695 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:52:01.807 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:52:01.836 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:52:01.890 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:52:01.912 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:52:01.933 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:52:02.121 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:52:02.294 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:52:02.425 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:52:02.527 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:52:02.595 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:52:02.808 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:52:02.918 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:52:03.016 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:52:03.049 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:52:03.093 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:52:03.169 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:52:03.219 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:52:03.255 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:52:03.255 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-09-10T09:52:03.255 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-09-10T09:52:03.255 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-09-10T09:52:03.255 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-09-10T09:52:03.255 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-09-10T09:52:03.255 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-09-10T09:52:03.256 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-09-10T09:52:03.256 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:03.259 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:52:03.302 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:52:03.335 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:52:03.363 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:52:03.434 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:52:03.435 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:52:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:52:03.452 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:03.452 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:52:03.452 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:03.486 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:52:03.515 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:52:03.551 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:52:03.630 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:52:03.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:52:03.705 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:52:03.733 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:52:03.807 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:52:03.873 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:52:03.947 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:52:03.951 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:52:04.097 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:52:04.169 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:52:04.175 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:52:04.721 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:52:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:52:04.773 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:52:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:52:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:52:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:52:04.849 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:52:04.849 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:52:04.849 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:04.882 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:52:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:52:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:52:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:52:04.997 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:52:05.025 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:52:05.025 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:05.025 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:52:05.025 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:52:05.026 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:52:05.026 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:52:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:52:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:52:05.264 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:52:05.289 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:52:05.289 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:05.290 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:52:05.290 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:52:05.290 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:52:05.290 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:52:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:52:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:52:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:52:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:52:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:52:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:52:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:52:05.756 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:52:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:52:05.781 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:52:05.781 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:05.781 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:52:05.781 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:52:05.781 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:52:05.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:52:05.895 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:52:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:52:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:52:05.953 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:52:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:52:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:52:06.017 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:52:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:52:06.103 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:52:06.213 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:52:06.234 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:52:06.268 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:52:06.277 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:52:06.306 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:52:06.340 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:52:06.348 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:52:06.380 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:52:06.419 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:52:06.446 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:52:06.449 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:52:06.483 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:52:06.516 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:52:06.559 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:52:06.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:52:06.592 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:52:06.634 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:52:06.665 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:52:06.683 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:52:06.693 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:52:06.731 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:52:06.762 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:52:06.870 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:52:06.922 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:52:07.149 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:52:07.184 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:52:07.215 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:52:07.280 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:52:07.307 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:52:07.685 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:52:07.735 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:52:07.769 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:52:07.872 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:52:07.925 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:52:07.970 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:52:08.061 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:52:08.096 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:52:08.285 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:52:08.310 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:52:08.390 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:52:08.411 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:52:08.442 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:52:09.075 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:52:09.204 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:52:09.457 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:52:09.484 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:52:09.593 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:52:09.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:52:09.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:52:09.837 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:52:09.837 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:09.837 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:52:09.837 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:52:09.837 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:52:09.837 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:09.895 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:52:09.907 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:52:10.017 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:52:10.077 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:52:10.488 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:52:10.572 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:52:10.601 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:52:10.634 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:52:10.959 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:52:11.005 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:52:11.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:52:11.080 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:52:11.081 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:52:11.106 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:52:11.107 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:11.107 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:52:11.107 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:52:11.107 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:52:11.107 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:11.110 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:52:11.146 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:52:11.149 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:52:11.173 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:52:11.173 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:11.173 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:52:11.173 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:11.225 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:52:11.249 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:52:11.250 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:11.250 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:52:11.250 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:52:11.250 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:52:11.250 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:11.576 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:52:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:52:11.600 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:11.600 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:52:11.600 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:52:11.600 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:52:11.600 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:11.652 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:52:11.683 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:52:11.754 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:52:11.784 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:52:14.150 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:52:14.189 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:52:14.761 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:52:14.801 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:52:14.875 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:52:14.875 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:52:14.875 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:52:14.875 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:14.902 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:52:16.447 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:52:16.489 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:52:16.530 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:52:16.561 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:52:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:52:16.836 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:52:16.866 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:52:16.896 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:52:16.951 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:52:16.951 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:52:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:52:16.967 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:16.967 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:52:16.967 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:17.590 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:17.624 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:17.645 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:18.459 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:52:18.485 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:52:18.486 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:18.486 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:52:18.486 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:52:18.486 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:52:18.486 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:18.577 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:52:18.663 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:52:18.709 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:52:18.715 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:52:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:52:18.733 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:18.733 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:52:18.733 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:52:18.733 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:52:18.733 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:18.813 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:52:18.850 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:52:18.875 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:52:18.901 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:52:18.901 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:52:18.994 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:52:18.994 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:52:19.086 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:52:19.111 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:52:19.118 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:19.118 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:19.185 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:52:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:52:19.208 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:19.208 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:52:19.208 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:52:19.208 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:52:19.208 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:19.305 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:19.305 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:52:19.306 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:52:19.363 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:52:19.381 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:52:19.426 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:52:19.450 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:19.612 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:52:19.661 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:52:19.738 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:52:19.834 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:52:19.973 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:52:20.019 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:52:20.051 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:52:20.079 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:52:20.458 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:52:20.487 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:52:20.518 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:52:20.580 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:52:20.610 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:52:21.650 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:52:21.686 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:52:22.648 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:52:22.670 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:52:22.771 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:52:22.932 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:52:22.965 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:52:22.989 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:52:22.989 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:22.989 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:52:22.989 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:52:22.989 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:52:22.989 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:23.032 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:52:23.043 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:52:23.133 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:52:23.188 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:52:24.118 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:52:24.151 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:52:24.175 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:52:24.175 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:24.175 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:52:24.175 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:52:24.175 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:52:24.175 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:24.218 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:52:24.241 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:52:24.241 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:24.241 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:52:24.242 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:24.286 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:52:24.309 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:52:24.309 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:24.309 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:52:24.310 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:52:24.310 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:52:24.310 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:24.612 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:52:24.635 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:52:24.635 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:24.636 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:52:24.636 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:52:24.636 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:52:24.636 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:52:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:52:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:52:25.040 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:52:25.041 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:52:25.042 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:52:25.053 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:52:25.054 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:52:25.055 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:52:25.056 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:52:25.057 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:52:25.058 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:52:25.059 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:52:25.060 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:52:25.061 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:52:25.062 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:52:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:52:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:52:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:52:25.064 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:52:27.333 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-09-10T09:52:27.334 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:27.506 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:52:27.523 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:52:27.523 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:27.523 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:52:27.523 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:27.950 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.951 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.952 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.953 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:52:27.954 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:52:27.955 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:52:27.956 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:52:27.957 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:52:27.958 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:52:27.959 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:52:27.960 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:52:27.961 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:52:27.962 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:52:27.962 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-09-10T09:52:28.356 DEBUG:teuthology.parallel:result is None 2024-09-10T09:52:29.043 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:52:29.070 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:52:29.070 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:29.070 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:52:29.070 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:52:29.070 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:52:29.070 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:29.183 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:52:29.224 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:52:29.265 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:52:29.297 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:52:29.302 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:52:29.324 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:52:29.325 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:52:29.325 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:29.325 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:52:29.325 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:52:29.326 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:52:29.326 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:29.516 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:52:29.563 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:52:29.585 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:52:29.615 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:52:29.652 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:52:29.786 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:52:29.809 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:52:29.809 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:29.809 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:52:29.809 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:52:29.809 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:52:29.810 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:29.925 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:52:29.983 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:52:30.047 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:52:30.202 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:30.242 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:52:30.242 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:30.262 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:30.296 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:52:30.377 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:52:30.475 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:52:30.616 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:52:30.662 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:52:30.694 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:52:30.721 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:52:31.096 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:52:31.168 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:52:31.190 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:52:31.196 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:52:31.227 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:52:31.231 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:52:31.291 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:52:31.314 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:52:31.318 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:52:31.343 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:52:31.369 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:52:31.403 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:52:31.403 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:52:31.504 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:52:31.504 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:52:31.579 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:52:31.613 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:52:31.618 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:31.618 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:31.798 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:31.798 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:52:31.882 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:52:31.945 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:32.424 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:52:32.469 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:52:33.485 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:52:33.511 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:52:33.621 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:52:33.789 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:52:33.822 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:52:33.846 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:52:33.846 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:33.846 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:52:33.846 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:52:33.846 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:52:33.847 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:33.898 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:52:33.910 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:52:34.011 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:52:34.063 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:52:35.037 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:52:35.067 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:52:35.090 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:52:35.091 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:35.091 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:52:35.091 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:52:35.091 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:52:35.091 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:35.134 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:52:35.157 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:52:35.157 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:35.157 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:52:35.157 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:35.201 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:52:35.224 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:52:35.224 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:35.225 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:52:35.225 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:52:35.225 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:52:35.225 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:52:35.568 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:52:35.568 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:52:35.568 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:52:35.568 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:52:35.568 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:52:35.568 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:52:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:52:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:52:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:52:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:52:37.446 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:52:37.447 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:52:37.448 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:52:37.449 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:52:37.450 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:52:37.451 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:52:37.452 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:52:37.453 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:52:37.454 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:52:37.455 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:52:37.456 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:52:37.457 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:52:37.458 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:52:37.459 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:52:37.460 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:52:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:39.981 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-09-10T09:52:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:39.981 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.982 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.983 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:52:39.984 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:52:39.985 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:39.986 INFO:teuthology.orchestra.run.smithi162.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.987 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:52:39.988 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:52:39.989 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:52:39.990 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:52:39.991 INFO:teuthology.orchestra.run.smithi162.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:52:39.992 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:52:39.993 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T09:52:40.373 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:52:40.381 DEBUG:teuthology.parallel:result is None 2024-09-10T09:52:40.415 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:52:40.456 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:52:40.487 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:52:40.515 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:52:40.717 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:52:40.761 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:52:40.791 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:52:40.821 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:52:40.867 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:52:41.482 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:41.523 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:41.543 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:52:42.434 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:52:42.528 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:52:42.569 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:52:42.653 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:52:42.689 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:52:42.714 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:52:42.749 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:52:42.749 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:52:42.825 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:52:42.825 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:52:42.909 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:52:42.934 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:52:42.939 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:42.939 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:43.127 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:52:43.127 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:52:43.211 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:52:43.278 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:52:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:52:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:52:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:52:48.621 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:52:48.622 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:52:48.623 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:52:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:52:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:52:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:52:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:52:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:52:48.638 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:52:48.639 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:52:48.640 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:52:48.641 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:52:48.642 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:52:48.643 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:52:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:52:48.645 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:52:48.646 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:52:48.647 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:52:48.648 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:52:51.213 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:52:51.213 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:52:51.214 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.215 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.216 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.217 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:52:51.218 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:52:51.219 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:52:51.220 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:52:51.221 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:52:51.222 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.223 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:52:51.224 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:52:51.225 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:52:51.226 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:52:51.226 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:52:51.226 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-09-10T09:52:51.626 DEBUG:teuthology.parallel:result is None 2024-09-10T09:52:51.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:52:51.785 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:52:51.813 INFO:teuthology.orchestra.run.smithi045.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:52:51.813 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:52:51.814 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:52:51.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:52:51.961 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:52:51.984 INFO:teuthology.orchestra.run.smithi047.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:52:51.984 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:52:51.984 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:52:51.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:52:52.137 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:52:52.163 INFO:teuthology.orchestra.run.smithi162.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:52:52.164 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:52:52.164 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:52:52.166 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T09:52:52.166 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:52:52.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:52:52.199 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:52:52.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:52:52.228 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:52:52.228 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:52:52.261 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T09:52:52.261 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:52:52.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:52:52.291 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:52:52.361 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:52:52.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:52:52.388 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:52:52.454 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:52:52.454 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:52:52.485 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:52:52.554 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T09:52:52.555 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:52:52.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:52:52.585 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:52:52.654 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:52:52.654 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:52:52.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:52:52.746 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:52:52.746 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:52:52.776 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:52:52.846 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T09:52:52.847 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:52:52.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:52:52.873 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:52:52.939 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:52:52.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:52:52.966 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:52:53.033 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:52:53.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:52:53.062 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:52:53.131 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T09:52:53.227 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2}} 2024-09-10T09:52:53.227 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:52:53.228 INFO:tasks.cephadm:Cluster fsid is 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:52:53.228 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T09:52:53.228 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.47', 'mon.c': '172.21.15.162'} 2024-09-10T09:52:53.228 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-09-10T09:52:53.228 INFO:tasks.cephadm:First mgr is x 2024-09-10T09:52:53.228 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T09:52:53.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-09-10T09:52:53.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-09-10T09:52:53.283 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname $(hostname -s) 2024-09-10T09:52:53.309 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T09:52:53.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:52:53.455 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T09:52:53.594 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:52:53.596 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T09:52:53.596 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T09:52:53.597 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:53.992 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:52 /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:53.992 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:54.383 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:52 /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:54.383 DEBUG:teuthology.orchestra.run.smithi162:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:54.750 INFO:teuthology.orchestra.run.smithi162.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:52 /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:54.751 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:54.772 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:54.792 DEBUG:teuthology.orchestra.run.smithi162:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:52:54.823 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T09:52:54.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:52:54.825 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:52:54.835 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:52:56.394 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:52:56.403 INFO:teuthology.orchestra.run.smithi162.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:52:56.468 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:53:27.919 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-10T09:53:27.919 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:53:27.920 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:53:27.920 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-09-10T09:53:27.920 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:53:27.920 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-09-10T09:53:27.920 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-10T09:53:44.324 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T09:53:44.324 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:53:44.324 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:53:44.324 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-09-10T09:53:44.324 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:53:44.324 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T09:53:44.324 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T09:53:44.344 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-10T09:53:44.345 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:53:44.345 INFO:teuthology.orchestra.run.smithi162.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:53:44.345 INFO:teuthology.orchestra.run.smithi162.stdout: "repo_digests": [ 2024-09-10T09:53:44.345 INFO:teuthology.orchestra.run.smithi162.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:53:44.345 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-10T09:53:44.345 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-10T09:53:44.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-09-10T09:53:44.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-09-10T09:53:44.433 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph 2024-09-10T09:53:44.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-09-10T09:53:44.486 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-09-10T09:53:44.515 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /etc/ceph 2024-09-10T09:53:44.541 INFO:tasks.cephadm:Writing seed config... 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T09:53:44.542 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T09:53:44.543 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-10T09:53:44.544 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T09:53:44.545 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-10T09:53:44.546 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:53:44.546 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T09:53:44.562 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 729b10fe-6f5a-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T09:53:44.562 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T09:53:44.604 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T09:53:44.647 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T09:53:44.647 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:53:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:53:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', '729b10fe-6f5a-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-09-10T09:53:44.857 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T09:53:44.857 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-09-10T09:53:44.882 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.2.2 2024-09-10T09:53:44.882 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-09-10T09:53:44.882 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-09-10T09:53:44.890 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T09:53:44.890 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T09:53:44.896 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T09:53:44.896 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-10T09:53:44.903 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-09-10T09:53:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-09-10T09:53:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-09-10T09:53:44.910 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-09-10T09:53:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 5.2.2 2024-09-10T09:53:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T09:53:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-09-10T09:53:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-09-10T09:53:44.939 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T09:53:44.939 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T09:53:44.945 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T09:53:44.946 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-10T09:53:44.952 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-09-10T09:53:44.959 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-09-10T09:53:44.959 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-09-10T09:53:44.959 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-09-10T09:53:44.959 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:53:44.959 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 139677507376128 on /run/cephadm/729b10fe-6f5a-11ef-bcea-c7b262605968.lock 2024-09-10T09:53:44.959 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 139677507376128 acquired on /run/cephadm/729b10fe-6f5a-11ef-bcea-c7b262605968.lock 2024-09-10T09:53:44.960 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-09-10T09:53:44.960 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-09-10T09:53:44.960 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-09-10T09:53:44.963 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-09-10T09:53:44.964 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-09-10T09:53:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T09:53:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T09:53:44.966 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2024-09-10T09:53:44.969 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T09:53:44.969 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T09:53:44.969 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T09:53:44.969 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T09:53:44.969 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-10T09:53:44.969 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T09:53:44.970 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-09-10T09:53:44.970 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-09-10T09:53:44.970 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T09:53:44.970 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T09:53:44.971 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:53:45.484 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T09:53:45.484 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:53:45.484 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T09:53:45.484 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T09:53:45.485 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T09:53:45.485 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T09:53:45.485 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T09:53:46.323 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T09:53:46.323 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T09:53:46.323 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T09:53:47.312 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-09-10T09:53:47.312 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-09-10T09:53:48.192 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQArF+BmTZIDIxAAMeyMHJYW92JX7wcaGa8KuQ== 2024-09-10T09:53:49.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQAsF+BmcwZbHxAAPmV8ot1hE+FxqWbGBtO3Dg== 2024-09-10T09:53:49.979 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQAtF+BmqUbkGhAA+lkh4bxgLhb5yBIJPY89rw== 2024-09-10T09:53:49.980 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-09-10T09:53:50.935 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T09:53:50.935 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:53:50.936 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-09-10T09:53:52.008 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-09-10T09:53:52.229 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T09:53:52.381 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T09:53:52.564 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968.target. 2024-09-10T09:53:52.564 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968.target. 2024-09-10T09:53:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a 2024-09-10T09:53:52.803 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T09:53:53.007 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968.target.wants/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@.service. 2024-09-10T09:53:53.467 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:53:53.467 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-10T09:53:53.474 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:53:53.474 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-10T09:53:53.474 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-10T09:53:53.474 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T09:53:53.475 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-09-10T09:53:53.475 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.564364s) 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T09:53:54.525 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.908+0000 7f811a6e1640 1 Processor -- start 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.908+0000 7f811a6e1640 1 -- start start 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.908+0000 7f811a6e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f8114107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.908+0000 7f811a6e1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81141080c0 con 0x7f81141076f0 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.908+0000 7f8113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f8114107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.908+0000 7f8113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f8114107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37820/0 (socket says 172.21.15.45:37820) 2024-09-10T09:53:54.526 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.908+0000 7f8113fff640 1 -- 172.21.15.45:0/1439131752 learned_addr learned my addr 172.21.15.45:0/1439131752 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.909+0000 7f8113fff640 1 -- 172.21.15.45:0/1439131752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141088f0 con 0x7f81141076f0 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.910+0000 7f8113fff640 1 --2- 172.21.15.45:0/1439131752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f8114107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8104009820 tx=0x7f810402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bc992e82ea6a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f8112ffd640 1 -- 172.21.15.45:0/1439131752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104037070 con 0x7f81141076f0 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f8112ffd640 1 -- 172.21.15.45:0/1439131752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8104032440 con 0x7f81141076f0 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f8112ffd640 1 -- 172.21.15.45:0/1439131752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104032740 con 0x7f81141076f0 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f811a6e1640 1 -- 172.21.15.45:0/1439131752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 msgr2=0x7f8114107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f811a6e1640 1 --2- 172.21.15.45:0/1439131752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f8114107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8104009820 tx=0x7f810402a270 comp rx=0 tx=0).stop 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f811a6e1640 1 -- 172.21.15.45:0/1439131752 shutdown_connections 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f811a6e1640 1 --2- 172.21.15.45:0/1439131752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f8114107af0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f811a6e1640 1 -- 172.21.15.45:0/1439131752 >> 172.21.15.45:0/1439131752 conn(0x7f8114102e80 msgr2=0x7f81141052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f811a6e1640 1 -- 172.21.15.45:0/1439131752 shutdown_connections 2024-09-10T09:53:54.527 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.911+0000 7f811a6e1640 1 -- 172.21.15.45:0/1439131752 wait complete. 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.912+0000 7f811a6e1640 1 Processor -- start 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.912+0000 7f811a6e1640 1 -- start start 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.912+0000 7f811a6e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f811419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.912+0000 7f811a6e1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8114108e90 con 0x7f81141076f0 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.912+0000 7f8113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f811419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.912+0000 7f8113fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f811419d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37824/0 (socket says 172.21.15.45:37824) 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.912+0000 7f8113fff640 1 -- 172.21.15.45:0/1229623687 learned_addr learned my addr 172.21.15.45:0/1229623687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f8113fff640 1 -- 172.21.15.45:0/1229623687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811419d670 con 0x7f81141076f0 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f8113fff640 1 --2- 172.21.15.45:0/1229623687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f811419d130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f81040090b0 tx=0x7f81040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f81117fa640 1 -- 172.21.15.45:0/1229623687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104041070 con 0x7f81141076f0 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f81117fa640 1 -- 172.21.15.45:0/1229623687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810403c7a0 con 0x7f81141076f0 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811419d900 con 0x7f81141076f0 2024-09-10T09:53:54.528 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f81117fa640 1 -- 172.21.15.45:0/1229623687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104037040 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a05f0 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f81117fa640 1 -- 172.21.15.45:0/1229623687 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8104044070 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.913+0000 7f81117fa640 1 -- 172.21.15.45:0/1229623687 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f810403d3f0 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.914+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81141094e0 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.917+0000 7f81117fa640 1 -- 172.21.15.45:0/1229623687 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8104046450 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.978+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f81141a0b40 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.978+0000 7f81117fa640 1 -- 172.21.15.45:0/1229623687 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f8104046630 con 0x7f81141076f0 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.979+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 msgr2=0x7f811419d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.979+0000 7f811a6e1640 1 --2- 172.21.15.45:0/1229623687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f811419d130 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f81040090b0 tx=0x7f81040046c0 comp rx=0 tx=0).stop 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.979+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 shutdown_connections 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.979+0000 7f811a6e1640 1 --2- 172.21.15.45:0/1229623687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81141076f0 0x7f811419d130 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.979+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 >> 172.21.15.45:0/1229623687 conn(0x7f8114102e80 msgr2=0x7f811407c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.979+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 shutdown_connections 2024-09-10T09:53:54.529 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:53.979+0000 7f811a6e1640 1 -- 172.21.15.45:0/1229623687 wait complete. 2024-09-10T09:53:54.530 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-09-10T09:53:54.530 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T09:53:55.623 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:53:55 smithi045 ceph-mon[30748]: from='client.? 172.21.15.45:0/1998070436' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T09:53:55.632 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.983+0000 7fc25baf4640 1 Processor -- start 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.984+0000 7fc25baf4640 1 -- start start 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.984+0000 7fc25baf4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc254107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.984+0000 7fc25baf4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc2541080c0 con 0x7fc2541076f0 2024-09-10T09:53:55.633 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.984+0000 7fc259890640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc254107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.984+0000 7fc259890640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc254107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37836/0 (socket says 172.21.15.45:37836) 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.984+0000 7fc259890640 1 -- 172.21.15.45:0/3748829868 learned_addr learned my addr 172.21.15.45:0/3748829868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.984+0000 7fc259890640 1 -- 172.21.15.45:0/3748829868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2541088f0 con 0x7fc2541076f0 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.985+0000 7fc259890640 1 --2- 172.21.15.45:0/3748829868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc254107af0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc250009820 tx=0x7fc25002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0cbd09806bd5da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.985+0000 7fc25888e640 1 -- 172.21.15.45:0/3748829868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc250037070 con 0x7fc2541076f0 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.985+0000 7fc25888e640 1 -- 172.21.15.45:0/3748829868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc250032440 con 0x7fc2541076f0 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.985+0000 7fc25888e640 1 -- 172.21.15.45:0/3748829868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc250032740 con 0x7fc2541076f0 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.986+0000 7fc25baf4640 1 -- 172.21.15.45:0/3748829868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 msgr2=0x7fc254107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.986+0000 7fc25baf4640 1 --2- 172.21.15.45:0/3748829868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc254107af0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc250009820 tx=0x7fc25002a270 comp rx=0 tx=0).stop 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.986+0000 7fc25baf4640 1 -- 172.21.15.45:0/3748829868 shutdown_connections 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.986+0000 7fc25baf4640 1 --2- 172.21.15.45:0/3748829868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc254107af0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.986+0000 7fc25baf4640 1 -- 172.21.15.45:0/3748829868 >> 172.21.15.45:0/3748829868 conn(0x7fc254102e80 msgr2=0x7fc2541052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.986+0000 7fc25baf4640 1 -- 172.21.15.45:0/3748829868 shutdown_connections 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.986+0000 7fc25baf4640 1 -- 172.21.15.45:0/3748829868 wait complete. 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc25baf4640 1 Processor -- start 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc25baf4640 1 -- start start 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc25baf4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc25407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc25baf4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc254108e90 con 0x7fc2541076f0 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc259890640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc25407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc259890640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc25407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37844/0 (socket says 172.21.15.45:37844) 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc259890640 1 -- 172.21.15.45:0/1998070436 learned_addr learned my addr 172.21.15.45:0/1998070436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.987+0000 7fc259890640 1 -- 172.21.15.45:0/1998070436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25407e090 con 0x7fc2541076f0 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc259890640 1 --2- 172.21.15.45:0/1998070436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc25407db50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc2500090b0 tx=0x7fc2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc250041070 con 0x7fc2541076f0 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25003c7a0 con 0x7fc2541076f0 2024-09-10T09:53:55.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25407c280 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc250037040 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25407c7c0 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc250044070 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.988+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc25003d3f0 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.989+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2541094e0 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:54.991+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc250046450 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.051+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc25407cac0 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.054+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fc250046630 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.054+0000 7fc24affd640 1 -- 172.21.15.45:0/1998070436 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc250056070 con 0x7fc2541076f0 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.055+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 msgr2=0x7fc25407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.055+0000 7fc25baf4640 1 --2- 172.21.15.45:0/1998070436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc25407db50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc2500090b0 tx=0x7fc2500046c0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.055+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 shutdown_connections 2024-09-10T09:53:55.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.055+0000 7fc25baf4640 1 --2- 172.21.15.45:0/1998070436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2541076f0 0x7fc25407db50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:55.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.055+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 >> 172.21.15.45:0/1998070436 conn(0x7fc254102e80 msgr2=0x7fc25418f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:55.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.055+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 shutdown_connections 2024-09-10T09:53:55.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:55.055+0000 7fc25baf4640 1 -- 172.21.15.45:0/1998070436 wait complete. 2024-09-10T09:53:55.637 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-09-10T09:53:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:53:57 smithi045 ceph-mon[30748]: from='client.? 172.21.15.45:0/1701167704' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:54:03.748 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.068+0000 7fa07e8f1640 1 Processor -- start 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.068+0000 7fa07e8f1640 1 -- start start 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.068+0000 7fa07e8f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa078107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.068+0000 7fa07e8f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0781080c0 con 0x7fa0781076f0 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.068+0000 7fa077fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa078107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.068+0000 7fa077fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa078107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37858/0 (socket says 172.21.15.45:37858) 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.068+0000 7fa077fff640 1 -- 172.21.15.45:0/4236157725 learned_addr learned my addr 172.21.15.45:0/4236157725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.069+0000 7fa077fff640 1 -- 172.21.15.45:0/4236157725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0781088f0 con 0x7fa0781076f0 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa077fff640 1 --2- 172.21.15.45:0/4236157725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa078107af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa068009820 tx=0x7fa06802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb879b1b7995ac2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa076ffd640 1 -- 172.21.15.45:0/4236157725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa068037070 con 0x7fa0781076f0 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa076ffd640 1 -- 172.21.15.45:0/4236157725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa068032900 con 0x7fa0781076f0 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa076ffd640 1 -- 172.21.15.45:0/4236157725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa068030850 con 0x7fa0781076f0 2024-09-10T09:54:03.749 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa07e8f1640 1 -- 172.21.15.45:0/4236157725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 msgr2=0x7fa078107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa07e8f1640 1 --2- 172.21.15.45:0/4236157725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa078107af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa068009820 tx=0x7fa06802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa07e8f1640 1 -- 172.21.15.45:0/4236157725 shutdown_connections 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa07e8f1640 1 --2- 172.21.15.45:0/4236157725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa078107af0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.070+0000 7fa07e8f1640 1 -- 172.21.15.45:0/4236157725 >> 172.21.15.45:0/4236157725 conn(0x7fa078102e80 msgr2=0x7fa0781052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.071+0000 7fa07e8f1640 1 -- 172.21.15.45:0/4236157725 shutdown_connections 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.071+0000 7fa07e8f1640 1 -- 172.21.15.45:0/4236157725 wait complete. 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.071+0000 7fa07e8f1640 1 Processor -- start 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.071+0000 7fa07e8f1640 1 -- start start 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa07e8f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa07819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa07e8f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810bca0 con 0x7fa0781076f0 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa077fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa07819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa077fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa07819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37864/0 (socket says 172.21.15.45:37864) 2024-09-10T09:54:03.750 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa077fff640 1 -- 172.21.15.45:0/1701167704 learned_addr learned my addr 172.21.15.45:0/1701167704 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa077fff640 1 -- 172.21.15.45:0/1701167704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07819bc80 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa077fff640 1 --2- 172.21.15.45:0/1701167704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa07819b740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa068009940 tx=0x7fa068032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa0757fa640 1 -- 172.21.15.45:0/1701167704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa068042070 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa0757fa640 1 -- 172.21.15.45:0/1701167704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa0680308c0 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa07819bf10 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.072+0000 7fa0757fa640 1 -- 172.21.15.45:0/1701167704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa068037040 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.073+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07819ec00 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.073+0000 7fa0757fa640 1 -- 172.21.15.45:0/1701167704 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa068045070 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.073+0000 7fa0757fa640 1 -- 172.21.15.45:0/1701167704 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa068047b70 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.074+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07810be20 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.076+0000 7fa0757fa640 1 -- 172.21.15.45:0/1701167704 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa0680327b0 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.138+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fa07819edd0 con 0x7fa0781076f0 2024-09-10T09:54:03.751 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.138+0000 7fa0757fa640 1 -- 172.21.15.45:0/1701167704 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fa06803c760 con 0x7fa0781076f0 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.139+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 msgr2=0x7fa07819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.139+0000 7fa07e8f1640 1 --2- 172.21.15.45:0/1701167704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa07819b740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa068009940 tx=0x7fa068032ce0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.139+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 shutdown_connections 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.139+0000 7fa07e8f1640 1 --2- 172.21.15.45:0/1701167704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0781076f0 0x7fa07819b740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.139+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 >> 172.21.15.45:0/1701167704 conn(0x7fa078102e80 msgr2=0x7fa078103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.139+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 shutdown_connections 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:53:57.139+0000 7fa07e8f1640 1 -- 172.21.15.45:0/1701167704 wait complete. 2024-09-10T09:54:03.752 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-09-10T09:54:04.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:03 smithi045 systemd[1]: Stopping Ceph mon.a for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T09:54:04.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a[30744]: 2024-09-10T09:54:04.556+0000 7f88a9d27640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:54:04.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a[30744]: 2024-09-10T09:54:04.556+0000 7f88a9d27640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T09:54:04.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:04 smithi045 podman[30988]: 2024-09-10 09:54:04.710257592 +0000 UTC m=+0.779035666 container died 8635bd404156df1d705ef4eecc9795dcd3f88be3bfc4327edea687e070571833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T09:54:05.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 podman[30988]: 2024-09-10 09:54:05.030143557 +0000 UTC m=+1.098921633 container cleanup 8635bd404156df1d705ef4eecc9795dcd3f88be3bfc4327edea687e070571833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:54:05.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 bash[30988]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a 2024-09-10T09:54:05.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 podman[31002]: 2024-09-10 09:54:05.19766974 +0000 UTC m=+0.479951517 container remove 8635bd404156df1d705ef4eecc9795dcd3f88be3bfc4327edea687e070571833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T09:54:05.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T09:54:05.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 systemd[1]: Stopped Ceph mon.a for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T09:54:05.725 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 systemd[1]: Starting Ceph mon.a for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T09:54:05.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 podman[31095]: 2024-09-10 09:54:05.615295795 +0000 UTC m=+0.100630872 container create ab3b8fe2420a38e21d48b516145674bf8fe66b23379607b0a03d295fd158fef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T09:54:05.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 podman[31095]: 2024-09-10 09:54:05.537315084 +0000 UTC m=+0.022650134 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:54:05.906 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T09:54:06.029 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 podman[31095]: 2024-09-10 09:54:05.826091392 +0000 UTC m=+0.311426428 container init ab3b8fe2420a38e21d48b516145674bf8fe66b23379607b0a03d295fd158fef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:54:06.029 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 podman[31095]: 2024-09-10 09:54:05.829553761 +0000 UTC m=+0.314888799 container start ab3b8fe2420a38e21d48b516145674bf8fe66b23379607b0a03d295fd158fef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T09:54:06.029 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: pidfile_write: ignore empty --pid-file 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: load: jerasure load: lrc 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: RocksDB version: 7.9.2 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Git sha 0 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: DB SUMMARY 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: DB Session ID: CDMQ729EEYI3TFVQ3Z3P 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: CURRENT file: CURRENT 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T09:54:06.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106050 ; 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.error_if_exists: 0 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.create_if_missing: 0 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.paranoid_checks: 1 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.env: 0x55650f816940 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.info_log: 0x5565107756e0 2024-09-10T09:54:06.031 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.statistics: (nil) 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.use_fsync: 0 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_log_file_size: 0 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.allow_fallocate: 1 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T09:54:06.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.use_direct_reads: 0 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.db_log_dir: 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.wal_dir: 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T09:54:06.034 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.write_buffer_manager: 0x556510784dc0 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.unordered_write: 0 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.row_cache: None 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.wal_filter: None 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.two_write_queues: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.wal_compression: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.atomic_flush: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.log_readahead_size: 0 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T09:54:06.036 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_background_jobs: 2 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_background_compactions: -1 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_subcompactions: 1 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T09:54:06.037 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_open_files: -1 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_background_flushes: -1 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Compression algorithms supported: 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kZSTD supported: 0 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kXpressCompression supported: 0 2024-09-10T09:54:06.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kBZip2Compression supported: 0 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kLZ4Compression supported: 1 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kZlibCompression supported: 1 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: kSnappyCompression supported: 1 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.merge_operator: 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_filter: None 2024-09-10T09:54:06.039 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556510775320) 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T09:54:06.040 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-09-10T09:54:06.041 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-09-10T09:54:06.041 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x5565107691f0 2024-09-10T09:54:06.041 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-09-10T09:54:06.041 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-09-10T09:54:06.041 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-09-10T09:54:06.041 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-09-10T09:54:06.041 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-09-10T09:54:06.042 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression: NoCompression 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T09:54:06.043 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.num_levels: 7 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:54:06.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T09:54:06.045 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T09:54:06.046 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.inplace_update_support: 0 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T09:54:06.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.bloom_locality: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.max_successive_merges: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.ttl: 2592000 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.enable_blob_files: false 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.min_blob_size: 0 2024-09-10T09:54:06.048 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4c93c9cc-aba9-4ebb-bdad-ab787238d84e 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725962045855779, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T09:54:06.049 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725962045856964, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 98895, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14508, "raw_average_key_size": 49, "raw_value_size": 91077, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725962045, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4c93c9cc-aba9-4ebb-bdad-ab787238d84e", "db_session_id": "CDMQ729EEYI3TFVQ3Z3P", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725962045857065, "job": 1, "event": "recovery_finished"} 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556510796e00 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: DB pointer 0x556510892000 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T09:54:06.050 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 100.50 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 100.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:54:06.051 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 28.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 28.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:54:06.052 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x5565107691f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???) e1 preinit fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???).mds e1 new map 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???).mds e1 print_map 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: btime 2024-09-10T09:53:53:416702+0000 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-09-10T09:54:06.053 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 bash[31095]: ab3b8fe2420a38e21d48b516145674bf8fe66b23379607b0a03d295fd158fef8 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 systemd[1]: Started Ceph mon.a for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: monmap epoch 1 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: last_changed 2024-09-10T09:53:50.319028+0000 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:54:06.054 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: min_mon_release 19 (squid) 2024-09-10T09:54:06.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: election_strategy: 1 2024-09-10T09:54:06.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:54:06.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: fsmap 2024-09-10T09:54:06.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T09:54:06.055 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:05 smithi045 ceph-mon[31114]: mgrmap e1: no daemons active 2024-09-10T09:54:06.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.395+0000 7f7ab3ea4640 1 Processor -- start 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.396+0000 7f7ab3ea4640 1 -- start start 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.396+0000 7f7ab3ea4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.396+0000 7f7ab3ea4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1080c0 con 0x7f7aac1076f0 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.396+0000 7f7ab1c40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.396+0000 7f7ab1c40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59352/0 (socket says 172.21.15.45:59352) 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.396+0000 7f7ab1c40640 1 -- 172.21.15.45:0/2248898412 learned_addr learned my addr 172.21.15.45:0/2248898412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.397+0000 7f7ab1c40640 1 -- 172.21.15.45:0/2248898412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac1088f0 con 0x7f7aac1076f0 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.399+0000 7f7ab1c40640 1 --2- 172.21.15.45:0/2248898412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009820 tx=0x7f7aa802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2babe4bd95be8c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.399+0000 7f7ab0c3e640 1 -- 172.21.15.45:0/2248898412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa8037070 con 0x7f7aac1076f0 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.399+0000 7f7ab0c3e640 1 -- 172.21.15.45:0/2248898412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7aa8032900 con 0x7f7aac1076f0 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.399+0000 7f7ab0c3e640 1 -- 172.21.15.45:0/2248898412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa8030850 con 0x7f7aac1076f0 2024-09-10T09:54:06.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/2248898412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 msgr2=0x7f7aac107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 --2- 172.21.15.45:0/2248898412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009820 tx=0x7f7aa802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:06.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/2248898412 shutdown_connections 2024-09-10T09:54:06.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 --2- 172.21.15.45:0/2248898412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac107af0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/2248898412 >> 172.21.15.45:0/2248898412 conn(0x7f7aac102e80 msgr2=0x7f7aac1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:06.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/2248898412 shutdown_connections 2024-09-10T09:54:06.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/2248898412 wait complete. 2024-09-10T09:54:06.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 Processor -- start 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 -- start start 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.400+0000 7f7ab3ea4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab3ea4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac10bca0 con 0x7f7aac1076f0 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab1c40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab1c40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59364/0 (socket says 172.21.15.45:59364) 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab1c40640 1 -- 172.21.15.45:0/3700357049 learned_addr learned my addr 172.21.15.45:0/3700357049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab1c40640 1 -- 172.21.15.45:0/3700357049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac19bc80 con 0x7f7aac1076f0 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab1c40640 1 --2- 172.21.15.45:0/3700357049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac19b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009940 tx=0x7f7aa8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7aa2ffd640 1 -- 172.21.15.45:0/3700357049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa8042070 con 0x7f7aac1076f0 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac19bf10 con 0x7f7aac1076f0 2024-09-10T09:54:06.944 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7aa2ffd640 1 -- 172.21.15.45:0/3700357049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7aa80308c0 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7aa2ffd640 1 -- 172.21.15.45:0/3700357049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa8037040 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac19ec00 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.401+0000 7f7aa2ffd640 1 -- 172.21.15.45:0/3700357049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7aa8045070 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.402+0000 7f7aa2ffd640 1 -- 172.21.15.45:0/3700357049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7aa8047bc0 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.402+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aac10be20 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.405+0000 7f7aa2ffd640 1 -- 172.21.15.45:0/3700357049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7aa80327b0 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.464+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7aac19f1f0 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.465+0000 7f7aa2ffd640 1 -- 172.21.15.45:0/3700357049 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7aa803c870 con 0x7f7aac1076f0 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.466+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 msgr2=0x7f7aac19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.466+0000 7f7ab3ea4640 1 --2- 172.21.15.45:0/3700357049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac19b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009940 tx=0x7f7aa8032ce0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.466+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 shutdown_connections 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.466+0000 7f7ab3ea4640 1 --2- 172.21.15.45:0/3700357049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7aac1076f0 0x7f7aac19b740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.466+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 >> 172.21.15.45:0/3700357049 conn(0x7f7aac102e80 msgr2=0x7f7aac103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:06.945 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.467+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 shutdown_connections 2024-09-10T09:54:06.946 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:06.467+0000 7f7ab3ea4640 1 -- 172.21.15.45:0/3700357049 wait complete. 2024-09-10T09:54:06.946 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T09:54:06.966 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:54:06.966 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-09-10T09:54:06.967 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T09:54:06.967 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T09:54:07.224 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.x 2024-09-10T09:54:07.224 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T09:54:07.384 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968.target.wants/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@.service. 2024-09-10T09:54:07.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:07 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3700357049' entity='client.admin' 2024-09-10T09:54:07.661 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:07 smithi045 podman[31323]: 2024-09-10 09:54:07.660603365 +0000 UTC m=+0.157155283 container create 5586f3d8900fc5466389422529797bd34bcddd63e6ef7a5417043badb5372805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T09:54:07.946 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:07 smithi045 podman[31323]: 2024-09-10 09:54:07.871296444 +0000 UTC m=+0.367848318 container init 5586f3d8900fc5466389422529797bd34bcddd63e6ef7a5417043badb5372805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T09:54:07.946 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:07 smithi045 podman[31323]: 2024-09-10 09:54:07.874052123 +0000 UTC m=+0.370604000 container start 5586f3d8900fc5466389422529797bd34bcddd63e6ef7a5417043badb5372805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:54:07.961 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:54:07.961 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-10T09:54:07.969 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:54:07.969 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-10T09:54:07.969 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-10T09:54:07.969 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T09:54:07.978 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:54:07.978 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-10T09:54:07.986 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:54:07.986 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-10T09:54:07.986 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-10T09:54:07.986 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T09:54:07.986 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-09-10T09:54:07.986 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-09-10T09:54:08.207 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:07 smithi045 bash[31323]: 5586f3d8900fc5466389422529797bd34bcddd63e6ef7a5417043badb5372805 2024-09-10T09:54:08.207 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:07 smithi045 systemd[1]: Started Ceph mgr.x for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T09:54:08.207 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:08.000+0000 7f0a6de73040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:54:08.207 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:08.061+0000 7f0a6de73040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:54:08.484 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:08.224+0000 7f0a6de73040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:54:08.484 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:08.294+0000 7f0a6de73040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:54:08.484 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:08.453+0000 7f0a6de73040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:54:08.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2801442158' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:54:08.845 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:08.610+0000 7f0a6de73040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:54:09.137 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:08.945+0000 7f0a6de73040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:54:09.137 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:09.008+0000 7f0a6de73040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:54:09.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:54:09.141 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "729b10fe-6f5a-11ef-bcea-c7b262605968", 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T09:54:09.142 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:54:09.143 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:53:53:416702+0000", 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:53:53.417331+0000", 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.412+0000 7fa20c814640 1 Processor -- start 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.412+0000 7fa20c814640 1 -- start start 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.413+0000 7fa20c814640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa208106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.413+0000 7fa20c814640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa20807be10 con 0x7fa208106010 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.413+0000 7fa20659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa208106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.413+0000 7fa20659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa208106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59396/0 (socket says 172.21.15.45:59396) 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.413+0000 7fa20659c640 1 -- 172.21.15.45:0/2138327455 learned_addr learned my addr 172.21.15.45:0/2138327455 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.413+0000 7fa20659c640 1 -- 172.21.15.45:0/2138327455 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa208106950 con 0x7fa208106010 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.414+0000 7fa20659c640 1 --2- 172.21.15.45:0/2138327455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa208106410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009820 tx=0x7fa1f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44200b5c1714a912 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.414+0000 7fa20559a640 1 -- 172.21.15.45:0/2138327455 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f8037070 con 0x7fa208106010 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.414+0000 7fa20559a640 1 -- 172.21.15.45:0/2138327455 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1f8032900 con 0x7fa208106010 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.414+0000 7fa20559a640 1 -- 172.21.15.45:0/2138327455 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f8030850 con 0x7fa208106010 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.415+0000 7fa20c814640 1 -- 172.21.15.45:0/2138327455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 msgr2=0x7fa208106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:09.154 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.415+0000 7fa20c814640 1 --2- 172.21.15.45:0/2138327455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa208106410 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009820 tx=0x7fa1f802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.415+0000 7fa20c814640 1 -- 172.21.15.45:0/2138327455 shutdown_connections 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.415+0000 7fa20c814640 1 --2- 172.21.15.45:0/2138327455 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa208106410 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.415+0000 7fa20c814640 1 -- 172.21.15.45:0/2138327455 >> 172.21.15.45:0/2138327455 conn(0x7fa208101e20 msgr2=0x7fa208104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.415+0000 7fa20c814640 1 -- 172.21.15.45:0/2138327455 shutdown_connections 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.415+0000 7fa20c814640 1 -- 172.21.15.45:0/2138327455 wait complete. 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20c814640 1 Processor -- start 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20c814640 1 -- start start 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20c814640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa20819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20c814640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa208107a50 con 0x7fa208106010 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa20819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa20819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59406/0 (socket says 172.21.15.45:59406) 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20659c640 1 -- 172.21.15.45:0/2801442158 learned_addr learned my addr 172.21.15.45:0/2801442158 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.416+0000 7fa20659c640 1 -- 172.21.15.45:0/2801442158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20819dea0 con 0x7fa208106010 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa20659c640 1 --2- 172.21.15.45:0/2801442158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa20819d960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80097f0 tx=0x7fa1f8030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:09.155 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa1f77fe640 1 -- 172.21.15.45:0/2801442158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f8042070 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa1f77fe640 1 -- 172.21.15.45:0/2801442158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1f8032ca0 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20819e130 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa1f77fe640 1 -- 172.21.15.45:0/2801442158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1f8037040 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2081a0e20 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa1f77fe640 1 -- 172.21.15.45:0/2801442158 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa1f8045070 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.417+0000 7fa1f77fe640 1 -- 172.21.15.45:0/2801442158 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa1f8047af0 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.418+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa208106410 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.420+0000 7fa1f77fe640 1 -- 172.21.15.45:0/2801442158 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa1f803db10 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.483+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa20802d940 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.483+0000 7fa1f77fe640 1 -- 172.21.15.45:0/2801442158 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fa1f803d430 con 0x7fa208106010 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.484+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 msgr2=0x7fa20819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.484+0000 7fa20c814640 1 --2- 172.21.15.45:0/2801442158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa20819d960 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80097f0 tx=0x7fa1f8030ec0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.484+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 shutdown_connections 2024-09-10T09:54:09.156 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.484+0000 7fa20c814640 1 --2- 172.21.15.45:0/2801442158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa208106010 0x7fa20819d960 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.484+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 >> 172.21.15.45:0/2801442158 conn(0x7fa208101e20 msgr2=0x7fa208194600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:09.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.484+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 shutdown_connections 2024-09-10T09:54:09.157 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:08.484+0000 7fa20c814640 1 -- 172.21.15.45:0/2801442158 wait complete. 2024-09-10T09:54:09.157 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-09-10T09:54:09.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:09.204+0000 7f0a6de73040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:54:10.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:10.135+0000 7f0a6de73040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:54:10.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:10.206+0000 7f0a6de73040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:54:11.240 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:10.965+0000 7f0a6de73040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:54:11.240 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.085+0000 7f0a6de73040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:54:11.531 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.238+0000 7f0a6de73040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:54:11.531 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.302+0000 7f0a6de73040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:54:11.531 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.458+0000 7f0a6de73040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:54:11.531 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.530+0000 7f0a6de73040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:54:11.827 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.592+0000 7f0a6de73040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:54:11.827 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.758+0000 7f0a6de73040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:54:11.827 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.826+0000 7f0a6de73040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:54:12.107 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.895+0000 7f0a6de73040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:54:12.107 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:11.981+0000 7f0a6de73040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:54:12.107 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:12.044+0000 7f0a6de73040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:54:12.107 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:12.106+0000 7f0a6de73040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:54:12.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:12 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/989979071' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:54:13.345 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:12.951+0000 7f0a6de73040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "729b10fe-6f5a-11ef-bcea-c7b262605968", 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:54:13.350 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:54:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:54:13.352 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:54:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:54:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:54:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:54:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:54:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:54:13.359 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:53:53:416702+0000", 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:54:13.360 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:13.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:53:53.417331+0000", 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.644+0000 7f4f2789b640 1 Processor -- start 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.644+0000 7f4f2789b640 1 -- start start 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.644+0000 7f4f2789b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201083a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.645+0000 7f4f2789b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2007be90 con 0x7f4f20105f90 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.645+0000 7f4f25637640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201083a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.645+0000 7f4f25637640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201083a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59416/0 (socket says 172.21.15.45:59416) 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.645+0000 7f4f25637640 1 -- 172.21.15.45:0/1556570967 learned_addr learned my addr 172.21.15.45:0/1556570967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.645+0000 7f4f25637640 1 -- 172.21.15.45:0/1556570967 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f201088e0 con 0x7f4f20105f90 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.646+0000 7f4f25637640 1 --2- 172.21.15.45:0/1556570967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201083a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009820 tx=0x7f4f1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb14969f9d3445b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.646+0000 7f4f17fff640 1 -- 172.21.15.45:0/1556570967 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c037070 con 0x7f4f20105f90 2024-09-10T09:54:13.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.646+0000 7f4f17fff640 1 -- 172.21.15.45:0/1556570967 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f1c032900 con 0x7f4f20105f90 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.646+0000 7f4f17fff640 1 -- 172.21.15.45:0/1556570967 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c030850 con 0x7f4f20105f90 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.647+0000 7f4f2789b640 1 -- 172.21.15.45:0/1556570967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 msgr2=0x7f4f201083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.647+0000 7f4f2789b640 1 --2- 172.21.15.45:0/1556570967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201083a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009820 tx=0x7f4f1c02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.647+0000 7f4f2789b640 1 -- 172.21.15.45:0/1556570967 shutdown_connections 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.647+0000 7f4f2789b640 1 --2- 172.21.15.45:0/1556570967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201083a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.647+0000 7f4f2789b640 1 -- 172.21.15.45:0/1556570967 >> 172.21.15.45:0/1556570967 conn(0x7f4f20101e20 msgr2=0x7f4f20104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.647+0000 7f4f2789b640 1 -- 172.21.15.45:0/1556570967 shutdown_connections 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.647+0000 7f4f2789b640 1 -- 172.21.15.45:0/1556570967 wait complete. 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.648+0000 7f4f2789b640 1 Processor -- start 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.648+0000 7f4f2789b640 1 -- start start 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.648+0000 7f4f2789b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.648+0000 7f4f2789b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201099e0 con 0x7f4f20105f90 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.648+0000 7f4f25637640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.648+0000 7f4f25637640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59428/0 (socket says 172.21.15.45:59428) 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.648+0000 7f4f25637640 1 -- 172.21.15.45:0/989979071 learned_addr learned my addr 172.21.15.45:0/989979071 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.649+0000 7f4f25637640 1 -- 172.21.15.45:0/989979071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f201a22f0 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.649+0000 7f4f25637640 1 --2- 172.21.15.45:0/989979071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201a1db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c031000 tx=0x7f4f1c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.649+0000 7f4f167fc640 1 -- 172.21.15.45:0/989979071 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c042070 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.649+0000 7f4f167fc640 1 -- 172.21.15.45:0/989979071 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f1c032be0 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.649+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f201a2580 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.649+0000 7f4f167fc640 1 -- 172.21.15.45:0/989979071 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f1c037040 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.649+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f201a5270 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.650+0000 7f4f167fc640 1 -- 172.21.15.45:0/989979071 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4f1c045070 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.650+0000 7f4f167fc640 1 -- 172.21.15.45:0/989979071 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4f1c047790 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.650+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f20106390 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.652+0000 7f4f167fc640 1 -- 172.21.15.45:0/989979071 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4f1c04c030 con 0x7f4f20105f90 2024-09-10T09:54:13.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.715+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4f2019b220 con 0x7f4f20105f90 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.715+0000 7f4f167fc640 1 -- 172.21.15.45:0/989979071 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f4f1c0328c0 con 0x7f4f20105f90 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.716+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 msgr2=0x7f4f201a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.716+0000 7f4f2789b640 1 --2- 172.21.15.45:0/989979071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201a1db0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c031000 tx=0x7f4f1c0326e0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.716+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 shutdown_connections 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.716+0000 7f4f2789b640 1 --2- 172.21.15.45:0/989979071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f20105f90 0x7f4f201a1db0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.716+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 >> 172.21.15.45:0/989979071 conn(0x7f4f20101e20 msgr2=0x7f4f20104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.716+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 shutdown_connections 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:12.716+0000 7f4f2789b640 1 -- 172.21.15.45:0/989979071 wait complete. 2024-09-10T09:54:13.365 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-09-10T09:54:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: Activating manager daemon x 2024-09-10T09:54:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: mgrmap e2: x(active, starting, since 0.00325298s) 2024-09-10T09:54:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:54:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:54:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:54:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:54:14.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:54:14.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: Manager daemon x is now available 2024-09-10T09:54:14.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:54:14.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:54:14.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' 2024-09-10T09:54:14.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' 2024-09-10T09:54:14.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:13 smithi045 ceph-mon[31114]: from='mgr.14100 172.21.15.45:0/3502618277' entity='mgr.x' 2024-09-10T09:54:15.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:14 smithi045 ceph-mon[31114]: mgrmap e3: x(active, since 1.00753s) 2024-09-10T09:54:15.998 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:15 smithi045 ceph-mon[31114]: mgrmap e4: x(active, since 2s) 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "729b10fe-6f5a-11ef-bcea-c7b262605968", 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:54:16.607 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:16.608 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:54:16.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:16.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:54:16.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:54:16.609 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:54:16.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:54:16.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:54:16.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:16.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:53:53:416702+0000", 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:54:16.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:53:53.417331+0000", 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.743+0000 7f59ffc75640 1 Processor -- start 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.743+0000 7f59ffc75640 1 -- start start 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.743+0000 7f59ffc75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f8107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:16.620 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.743+0000 7f59ffc75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59f81080c0 con 0x7f59f81076f0 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.743+0000 7f59fda11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f8107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.744+0000 7f59fda11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f8107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47836/0 (socket says 172.21.15.45:47836) 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.744+0000 7f59fda11640 1 -- 172.21.15.45:0/2393001182 learned_addr learned my addr 172.21.15.45:0/2393001182 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.744+0000 7f59fda11640 1 -- 172.21.15.45:0/2393001182 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f81088f0 con 0x7f59f81076f0 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.745+0000 7f59fda11640 1 --2- 172.21.15.45:0/2393001182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f8107af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009820 tx=0x7f59f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfee2da19c6d6dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.745+0000 7f59fca0f640 1 -- 172.21.15.45:0/2393001182 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f4037070 con 0x7f59f81076f0 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.745+0000 7f59fca0f640 1 -- 172.21.15.45:0/2393001182 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59f4032900 con 0x7f59f81076f0 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.745+0000 7f59fca0f640 1 -- 172.21.15.45:0/2393001182 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f4030850 con 0x7f59f81076f0 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 -- 172.21.15.45:0/2393001182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 msgr2=0x7f59f8107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 --2- 172.21.15.45:0/2393001182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f8107af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009820 tx=0x7f59f402a270 comp rx=0 tx=0).stop 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 -- 172.21.15.45:0/2393001182 shutdown_connections 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 --2- 172.21.15.45:0/2393001182 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f8107af0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:16.621 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 -- 172.21.15.45:0/2393001182 >> 172.21.15.45:0/2393001182 conn(0x7f59f8102e80 msgr2=0x7f59f81052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 -- 172.21.15.45:0/2393001182 shutdown_connections 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 -- 172.21.15.45:0/2393001182 wait complete. 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.746+0000 7f59ffc75640 1 Processor -- start 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59ffc75640 1 -- start start 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59ffc75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59ffc75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59f810bca0 con 0x7f59f81076f0 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59fda11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59fda11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47846/0 (socket says 172.21.15.45:47846) 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59fda11640 1 -- 172.21.15.45:0/1202012819 learned_addr learned my addr 172.21.15.45:0/1202012819 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59fda11640 1 -- 172.21.15.45:0/1202012819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f807e090 con 0x7f59f81076f0 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59fda11640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f807db50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59f40097f0 tx=0x7f59f4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59eeffd640 1 -- 172.21.15.45:0/1202012819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f4037040 con 0x7f59f81076f0 2024-09-10T09:54:16.622 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59eeffd640 1 -- 172.21.15.45:0/1202012819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59f4040070 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f807c280 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59eeffd640 1 -- 172.21.15.45:0/1202012819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59f403a390 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.747+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f807c760 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.748+0000 7f59eeffd640 1 -- 172.21.15.45:0/1202012819 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f59f403a5e0 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.748+0000 7f59eeffd640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f59e4038f70 0x7f59e403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.748+0000 7f59eeffd640 1 -- 172.21.15.45:0/1202012819 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59f406c090 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.748+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f810be20 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.748+0000 7f59fd210640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f59e4038f70 0x7f59e403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.749+0000 7f59fd210640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f59e4038f70 0x7f59e403b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098c0 tx=0x7f59e8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.754+0000 7f59eeffd640 1 -- 172.21.15.45:0/1202012819 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59f406bdf0 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.997+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f59f807cf30 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:15.997+0000 7f59eeffd640 1 -- 172.21.15.45:0/1202012819 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f59f403a8d0 con 0x7f59f81076f0 2024-09-10T09:54:16.623 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f59e4038f70 msgr2=0x7f59e403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f59e4038f70 0x7f59e403b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59e80098c0 tx=0x7f59e8006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 msgr2=0x7f59f807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f807db50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f59f40097f0 tx=0x7f59f4030d80 comp rx=0 tx=0).stop 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 shutdown_connections 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f59e4038f70 0x7f59e403b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 --2- 172.21.15.45:0/1202012819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59f81076f0 0x7f59f807db50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 >> 172.21.15.45:0/1202012819 conn(0x7f59f8102e80 msgr2=0x7f59f8103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:16.624 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.000+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 shutdown_connections 2024-09-10T09:54:16.625 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.001+0000 7f59ffc75640 1 -- 172.21.15.45:0/1202012819 wait complete. 2024-09-10T09:54:16.625 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-09-10T09:54:17.169 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:16 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1202012819' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:54:18.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:17 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/4129585909' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T09:54:18.310 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T09:54:18.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.984+0000 7f8b339a2640 1 Processor -- start 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.984+0000 7f8b339a2640 1 -- start start 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.984+0000 7f8b339a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.984+0000 7f8b339a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c106fe0 con 0x7f8b2c106610 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.984+0000 7f8b3173e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.984+0000 7f8b3173e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47860/0 (socket says 172.21.15.45:47860) 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.984+0000 7f8b3173e640 1 -- 172.21.15.45:0/2921421359 learned_addr learned my addr 172.21.15.45:0/2921421359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.985+0000 7f8b3173e640 1 -- 172.21.15.45:0/2921421359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c107810 con 0x7f8b2c106610 2024-09-10T09:54:18.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.985+0000 7f8b3173e640 1 --2- 172.21.15.45:0/2921421359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c106a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009820 tx=0x7f8b2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8607ed3cccb2de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.985+0000 7f8b2bfff640 1 -- 172.21.15.45:0/2921421359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b24037070 con 0x7f8b2c106610 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.985+0000 7f8b2bfff640 1 -- 172.21.15.45:0/2921421359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b24032900 con 0x7f8b2c106610 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.985+0000 7f8b2bfff640 1 -- 172.21.15.45:0/2921421359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b24030850 con 0x7f8b2c106610 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.986+0000 7f8b339a2640 1 -- 172.21.15.45:0/2921421359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 msgr2=0x7f8b2c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.986+0000 7f8b339a2640 1 --2- 172.21.15.45:0/2921421359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c106a10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8b24009820 tx=0x7f8b2402a270 comp rx=0 tx=0).stop 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.986+0000 7f8b339a2640 1 -- 172.21.15.45:0/2921421359 shutdown_connections 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.986+0000 7f8b339a2640 1 --2- 172.21.15.45:0/2921421359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c106a10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.986+0000 7f8b339a2640 1 -- 172.21.15.45:0/2921421359 >> 172.21.15.45:0/2921421359 conn(0x7f8b2c101da0 msgr2=0x7f8b2c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.986+0000 7f8b339a2640 1 -- 172.21.15.45:0/2921421359 shutdown_connections 2024-09-10T09:54:18.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.986+0000 7f8b339a2640 1 -- 172.21.15.45:0/2921421359 wait complete. 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b339a2640 1 Processor -- start 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b339a2640 1 -- start start 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b339a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b339a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2c10abc0 con 0x7f8b2c106610 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b3173e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b3173e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47866/0 (socket says 172.21.15.45:47866) 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b3173e640 1 -- 172.21.15.45:0/4129585909 learned_addr learned my addr 172.21.15.45:0/4129585909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b3173e640 1 -- 172.21.15.45:0/4129585909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2c19bc90 con 0x7f8b2c106610 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.987+0000 7f8b3173e640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c19b750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b240097f0 tx=0x7f8b24030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.988+0000 7f8b2a7fc640 1 -- 172.21.15.45:0/4129585909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b24037040 con 0x7f8b2c106610 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.988+0000 7f8b2a7fc640 1 -- 172.21.15.45:0/4129585909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b24040070 con 0x7f8b2c106610 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.988+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2c19bf20 con 0x7f8b2c106610 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.988+0000 7f8b2a7fc640 1 -- 172.21.15.45:0/4129585909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b2403a390 con 0x7f8b2c106610 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.988+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b2c19ec10 con 0x7f8b2c106610 2024-09-10T09:54:18.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.989+0000 7f8b2a7fc640 1 -- 172.21.15.45:0/4129585909 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8b2403a630 con 0x7f8b2c106610 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.989+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2c106a90 con 0x7f8b2c106610 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.989+0000 7f8b2a7fc640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f8b14038f20 0x7f8b1403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.989+0000 7f8b2a7fc640 1 -- 172.21.15.45:0/4129585909 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8b2406c110 con 0x7f8b2c106610 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.989+0000 7f8b30f3d640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f8b14038f20 0x7f8b1403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.989+0000 7f8b30f3d640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f8b14038f20 0x7f8b1403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000ad30 tx=0x7f8b200093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:16.994+0000 7f8b2a7fc640 1 -- 172.21.15.45:0/4129585909 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b2403a9d0 con 0x7f8b2c106610 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.167+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8b2c19c3a0 con 0x7f8b2c106610 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.169+0000 7f8b2a7fc640 1 -- 172.21.15.45:0/4129585909 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f8b2406be70 con 0x7f8b2c106610 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f8b14038f20 msgr2=0x7f8b1403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f8b14038f20 0x7f8b1403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000ad30 tx=0x7f8b200093f0 comp rx=0 tx=0).stop 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 msgr2=0x7f8b2c19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c19b750 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8b240097f0 tx=0x7f8b24030d80 comp rx=0 tx=0).stop 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 shutdown_connections 2024-09-10T09:54:18.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f8b14038f20 0x7f8b1403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:18.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 --2- 172.21.15.45:0/4129585909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b2c106610 0x7f8b2c19b750 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:18.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 >> 172.21.15.45:0/4129585909 conn(0x7f8b2c101da0 msgr2=0x7f8b2c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:18.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 shutdown_connections 2024-09-10T09:54:18.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:17.171+0000 7f8b339a2640 1 -- 172.21.15.45:0/4129585909 wait complete. 2024-09-10T09:54:18.316 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-09-10T09:54:19.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3463119090' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T09:54:19.223 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ignoring --setuser ceph since I am not root 2024-09-10T09:54:19.223 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ignoring --setgroup ceph since I am not root 2024-09-10T09:54:19.223 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:19.162+0000 7fcc89a08040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:54:19.223 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:19.221+0000 7fcc89a08040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:54:19.599 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:19.376+0000 7fcc89a08040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:54:19.600 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:19.445+0000 7fcc89a08040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:54:19.600 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:19.598+0000 7fcc89a08040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:54:19.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.731+0000 7f1a6d6dc640 1 Processor -- start 2024-09-10T09:54:19.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.731+0000 7f1a6d6dc640 1 -- start start 2024-09-10T09:54:19.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.731+0000 7f1a6d6dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a68107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.731+0000 7f1a6d6dc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681080c0 con 0x7f1a681076f0 2024-09-10T09:54:19.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.731+0000 7f1a66ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a68107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.731+0000 7f1a66ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a68107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47882/0 (socket says 172.21.15.45:47882) 2024-09-10T09:54:19.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.731+0000 7f1a66ffd640 1 -- 172.21.15.45:0/4236633517 learned_addr learned my addr 172.21.15.45:0/4236633517 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.732+0000 7f1a66ffd640 1 -- 172.21.15.45:0/4236633517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a681088f0 con 0x7f1a681076f0 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.732+0000 7f1a66ffd640 1 --2- 172.21.15.45:0/4236633517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a68107af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009820 tx=0x7f1a5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d60de5d8fd0e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a65ffb640 1 -- 172.21.15.45:0/4236633517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a58037070 con 0x7f1a681076f0 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a65ffb640 1 -- 172.21.15.45:0/4236633517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a58032900 con 0x7f1a681076f0 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a65ffb640 1 -- 172.21.15.45:0/4236633517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a58030850 con 0x7f1a681076f0 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/4236633517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 msgr2=0x7f1a68107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a6d6dc640 1 --2- 172.21.15.45:0/4236633517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a68107af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009820 tx=0x7f1a5802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/4236633517 shutdown_connections 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a6d6dc640 1 --2- 172.21.15.45:0/4236633517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a68107af0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/4236633517 >> 172.21.15.45:0/4236633517 conn(0x7f1a68102e80 msgr2=0x7f1a681052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/4236633517 shutdown_connections 2024-09-10T09:54:19.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.733+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/4236633517 wait complete. 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.734+0000 7f1a6d6dc640 1 Processor -- start 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.734+0000 7f1a6d6dc640 1 -- start start 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.734+0000 7f1a6d6dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a6819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.734+0000 7f1a6d6dc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6810bca0 con 0x7f1a681076f0 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.734+0000 7f1a66ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a6819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.734+0000 7f1a66ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a6819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47884/0 (socket says 172.21.15.45:47884) 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.734+0000 7f1a66ffd640 1 -- 172.21.15.45:0/3463119090 learned_addr learned my addr 172.21.15.45:0/3463119090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.735+0000 7f1a66ffd640 1 -- 172.21.15.45:0/3463119090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6819bc80 con 0x7f1a681076f0 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.735+0000 7f1a66ffd640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a6819b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1a580097f0 tx=0x7f1a58030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.735+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a58037040 con 0x7f1a681076f0 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.735+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a58040070 con 0x7f1a681076f0 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.735+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6819bf10 con 0x7f1a681076f0 2024-09-10T09:54:19.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.735+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a5803a390 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.735+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6819ec00 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.736+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1a5803a630 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.736+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a68107b70 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.736+0000 7f1a4ffff640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f1a50038f20 0x7f1a5003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.736+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1a5806c0e0 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.737+0000 7f1a667fc640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f1a50038f20 0x7f1a5003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.737+0000 7f1a667fc640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f1a50038f20 0x7f1a5003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098c0 tx=0x7f1a5c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.741+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a5803a9d0 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:18.966+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f1a6819c390 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.018+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1a5806be40 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.019+0000 7f1a4ffff640 1 -- 172.21.15.45:0/3463119090 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f1a5802fd60 con 0x7f1a681076f0 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.020+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f1a50038f20 msgr2=0x7f1a5003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.020+0000 7f1a6d6dc640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f1a50038f20 0x7f1a5003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098c0 tx=0x7f1a5c006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.668 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.020+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 msgr2=0x7f1a6819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.020+0000 7f1a6d6dc640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a6819b740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1a580097f0 tx=0x7f1a58030d80 comp rx=0 tx=0).stop 2024-09-10T09:54:19.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.020+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 shutdown_connections 2024-09-10T09:54:19.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.021+0000 7f1a6d6dc640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7f1a50038f20 0x7f1a5003b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.021+0000 7f1a6d6dc640 1 --2- 172.21.15.45:0/3463119090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a681076f0 0x7f1a6819b740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.021+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 >> 172.21.15.45:0/3463119090 conn(0x7f1a68102e80 msgr2=0x7f1a68103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.021+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 shutdown_connections 2024-09-10T09:54:19.669 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:19.021+0000 7f1a6d6dc640 1 -- 172.21.15.45:0/3463119090 wait complete. 2024-09-10T09:54:19.900 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:19.748+0000 7fcc89a08040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:54:20.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:20 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3463119090' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T09:54:20.272 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:20 smithi045 ceph-mon[31114]: mgrmap e5: x(active, since 6s) 2024-09-10T09:54:20.273 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:20.083+0000 7fcc89a08040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:54:20.273 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:20.147+0000 7fcc89a08040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:54:20.650 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:20.344+0000 7fcc89a08040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:54:20.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:54:20.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T09:54:20.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:54:20.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T09:54:20.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T09:54:20.957 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.064+0000 7fcb1003d640 1 Processor -- start 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.064+0000 7fcb1003d640 1 -- start start 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.065+0000 7fcb1003d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb08106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.065+0000 7fcb1003d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08107000 con 0x7fcb08106630 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.065+0000 7fcb0ddd9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb08106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.065+0000 7fcb0ddd9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb08106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47898/0 (socket says 172.21.15.45:47898) 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.065+0000 7fcb0ddd9640 1 -- 172.21.15.45:0/2757505097 learned_addr learned my addr 172.21.15.45:0/2757505097 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.065+0000 7fcb0ddd9640 1 -- 172.21.15.45:0/2757505097 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb08107830 con 0x7fcb08106630 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.066+0000 7fcb0ddd9640 1 --2- 172.21.15.45:0/2757505097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb08106a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009820 tx=0x7fcb0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f5847409c0ff2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.066+0000 7fcb0cdd7640 1 -- 172.21.15.45:0/2757505097 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb04037070 con 0x7fcb08106630 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.066+0000 7fcb0cdd7640 1 -- 172.21.15.45:0/2757505097 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb04032900 con 0x7fcb08106630 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.066+0000 7fcb0cdd7640 1 -- 172.21.15.45:0/2757505097 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb04030850 con 0x7fcb08106630 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.067+0000 7fcb1003d640 1 -- 172.21.15.45:0/2757505097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 msgr2=0x7fcb08106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:20.958 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.067+0000 7fcb1003d640 1 --2- 172.21.15.45:0/2757505097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb08106a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009820 tx=0x7fcb0402a270 comp rx=0 tx=0).stop 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.067+0000 7fcb1003d640 1 -- 172.21.15.45:0/2757505097 shutdown_connections 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.067+0000 7fcb1003d640 1 --2- 172.21.15.45:0/2757505097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb08106a30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.067+0000 7fcb1003d640 1 -- 172.21.15.45:0/2757505097 >> 172.21.15.45:0/2757505097 conn(0x7fcb08101e20 msgr2=0x7fcb08104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.067+0000 7fcb1003d640 1 -- 172.21.15.45:0/2757505097 shutdown_connections 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.067+0000 7fcb1003d640 1 -- 172.21.15.45:0/2757505097 wait complete. 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb1003d640 1 Processor -- start 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb1003d640 1 -- start start 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb1003d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb0819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb1003d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0810abe0 con 0x7fcb08106630 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb0ddd9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb0819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb0ddd9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb0819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47906/0 (socket says 172.21.15.45:47906) 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb0ddd9640 1 -- 172.21.15.45:0/1475658758 learned_addr learned my addr 172.21.15.45:0/1475658758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:20.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb0ddd9640 1 -- 172.21.15.45:0/1475658758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0819bd10 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.068+0000 7fcb0ddd9640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb0819b7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb040097f0 tx=0x7fcb04030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.069+0000 7fcafeffd640 1 -- 172.21.15.45:0/1475658758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb04042070 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.069+0000 7fcafeffd640 1 -- 172.21.15.45:0/1475658758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb04032ca0 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.069+0000 7fcafeffd640 1 -- 172.21.15.45:0/1475658758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb04037040 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.069+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0819bfa0 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.069+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0819ec90 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.070+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0810ad60 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.070+0000 7fcafeffd640 1 -- 172.21.15.45:0/1475658758 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fcb04045070 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.070+0000 7fcafeffd640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 0x7fcaf403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.070+0000 7fcafeffd640 1 -- 172.21.15.45:0/1475658758 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcb0406be70 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.070+0000 7fcb0d5d8640 1 -- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 msgr2=0x7fcaf403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/986599852 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.070+0000 7fcb0d5d8640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 0x7fcaf403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.075+0000 7fcafeffd640 1 -- 172.21.15.45:0/1475658758 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb0404b170 con 0x7fcb08106630 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.271+0000 7fcb0d5d8640 1 -- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 msgr2=0x7fcaf403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/986599852 2024-09-10T09:54:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.271+0000 7fcb0d5d8640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 0x7fcaf403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.302+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcb0819f320 con 0x7fcb08106630 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.302+0000 7fcafeffd640 1 -- 172.21.15.45:0/1475658758 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcb0406b890 con 0x7fcb08106630 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 msgr2=0x7fcaf403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 0x7fcaf403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 msgr2=0x7fcb0819b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb0819b7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb040097f0 tx=0x7fcb04030ec0 comp rx=0 tx=0).stop 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 shutdown_connections 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fcaf4038fc0 0x7fcaf403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 --2- 172.21.15.45:0/1475658758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb08106630 0x7fcb0819b7d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 >> 172.21.15.45:0/1475658758 conn(0x7fcb08101e20 msgr2=0x7fcb08102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.304+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 shutdown_connections 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:20.305+0000 7fcb1003d640 1 -- 172.21.15.45:0/1475658758 wait complete. 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-09-10T09:54:20.961 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-09-10T09:54:21.268 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:21 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1475658758' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T09:54:21.555 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:21 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:21.275+0000 7fcc89a08040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:54:21.555 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:21 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:21.347+0000 7fcc89a08040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:54:22.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.091+0000 7fcc89a08040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:54:22.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.211+0000 7fcc89a08040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:54:22.579 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.360+0000 7fcc89a08040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:54:22.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.423+0000 7fcc89a08040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:54:22.580 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.578+0000 7fcc89a08040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:54:22.877 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.649+0000 7fcc89a08040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:54:22.877 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.711+0000 7fcc89a08040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:54:22.877 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.876+0000 7fcc89a08040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:54:23.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:22 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:22.943+0000 7fcc89a08040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:54:23.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:23 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:23.012+0000 7fcc89a08040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:54:23.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:23 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:23.098+0000 7fcc89a08040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:54:23.650 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:23 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:23.160+0000 7fcc89a08040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:54:23.650 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:23 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:23.221+0000 7fcc89a08040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:54:24.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:24.045+0000 7fcc89a08040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:54:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: Active manager daemon x restarted 2024-09-10T09:54:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: Activating manager daemon x 2024-09-10T09:54:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T09:54:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: mgrmap e6: x(active, starting, since 0.003174s) 2024-09-10T09:54:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:54:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: Manager daemon x is now available 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: Found migration_current of "None". Setting to last migration. 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:54:24.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:24 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:54:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:54:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T09:54:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T09:54:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:54:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.347+0000 7fac91832640 1 Processor -- start 2024-09-10T09:54:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.347+0000 7fac91832640 1 -- start start 2024-09-10T09:54:25.614 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.347+0000 7fac91832640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c10a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.347+0000 7fac91832640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c07be30 con 0x7fac8c1081e0 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.348+0000 7fac8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c10a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.348+0000 7fac8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c10a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47912/0 (socket says 172.21.15.45:47912) 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.348+0000 7fac8affd640 1 -- 172.21.15.45:0/2008396179 learned_addr learned my addr 172.21.15.45:0/2008396179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.348+0000 7fac8affd640 1 -- 172.21.15.45:0/2008396179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c10ab10 con 0x7fac8c1081e0 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.349+0000 7fac8affd640 1 --2- 172.21.15.45:0/2008396179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c10a5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009820 tx=0x7fac7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaaa7fcc94b25975 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.349+0000 7fac89ffb640 1 -- 172.21.15.45:0/2008396179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac7c037070 con 0x7fac8c1081e0 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.349+0000 7fac89ffb640 1 -- 172.21.15.45:0/2008396179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac7c032900 con 0x7fac8c1081e0 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.349+0000 7fac89ffb640 1 -- 172.21.15.45:0/2008396179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac7c030850 con 0x7fac8c1081e0 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.350+0000 7fac91832640 1 -- 172.21.15.45:0/2008396179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 msgr2=0x7fac8c10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.350+0000 7fac91832640 1 --2- 172.21.15.45:0/2008396179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c10a5d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009820 tx=0x7fac7c02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.350+0000 7fac91832640 1 -- 172.21.15.45:0/2008396179 shutdown_connections 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.350+0000 7fac91832640 1 --2- 172.21.15.45:0/2008396179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c10a5d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.615 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.350+0000 7fac91832640 1 -- 172.21.15.45:0/2008396179 >> 172.21.15.45:0/2008396179 conn(0x7fac8c101da0 msgr2=0x7fac8c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.350+0000 7fac91832640 1 -- 172.21.15.45:0/2008396179 shutdown_connections 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.350+0000 7fac91832640 1 -- 172.21.15.45:0/2008396179 wait complete. 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.351+0000 7fac91832640 1 Processor -- start 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.351+0000 7fac91832640 1 -- start start 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.351+0000 7fac91832640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.351+0000 7fac91832640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c10bb50 con 0x7fac8c1081e0 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.351+0000 7fac8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.351+0000 7fac8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47918/0 (socket says 172.21.15.45:47918) 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.351+0000 7fac8affd640 1 -- 172.21.15.45:0/1542715943 learned_addr learned my addr 172.21.15.45:0/1542715943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.352+0000 7fac8affd640 1 -- 172.21.15.45:0/1542715943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c19bc80 con 0x7fac8c1081e0 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.352+0000 7fac8affd640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c19b740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac7c032ce0 tx=0x7fac7c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.616 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.352+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac7c042070 con 0x7fac8c1081e0 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.352+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c19bf10 con 0x7fac8c1081e0 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.352+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac7c0308c0 con 0x7fac8c1081e0 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.352+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac7c037040 con 0x7fac8c1081e0 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.352+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c19ec00 con 0x7fac8c1081e0 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.353+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fac7c045070 con 0x7fac8c1081e0 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.353+0000 7fac90830640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 0x7fac7403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.353+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 --> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac7403bb40 con 0x7fac74038f70 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.353+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac7c06c1d0 con 0x7fac8c1081e0 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.353+0000 7fac8a7fc640 1 -- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 msgr2=0x7fac7403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/986599852 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.353+0000 7fac8a7fc640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 0x7fac7403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.554+0000 7fac8a7fc640 1 -- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 msgr2=0x7fac7403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/986599852 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.554+0000 7fac8a7fc640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 0x7fac7403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:54:25.617 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.955+0000 7fac8a7fc640 1 -- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 msgr2=0x7fac7403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/986599852 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:21.955+0000 7fac8a7fc640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 0x7fac7403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:22.756+0000 7fac8a7fc640 1 -- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 msgr2=0x7fac7403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/986599852 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:22.756+0000 7fac8a7fc640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 0x7fac7403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:24.048+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fac7c06ad50 con 0x7fac8c1081e0 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:24.048+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 msgr2=0x7fac7403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:24.048+0000 7fac90830640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 0x7fac7403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.049+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fac7c03cd00 con 0x7fac8c1081e0 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.049+0000 7fac90830640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fac7403c8a0 0x7fac7403ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.050+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac7403bb40 con 0x7fac7403c8a0 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.050+0000 7fac8a7fc640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fac7403c8a0 0x7fac7403ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.051+0000 7fac8a7fc640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fac7403c8a0 0x7fac7403ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fac80003980 tx=0x7fac80006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.051+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fac7403bb40 con 0x7fac7403c8a0 2024-09-10T09:54:25.618 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.058+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fac8c108640 con 0x7fac7403c8a0 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.058+0000 7fac90830640 1 -- 172.21.15.45:0/1542715943 <== mgr.14118 v2:172.21.15.45:6800/1958743370 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fac8c108640 con 0x7fac7403c8a0 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fac7403c8a0 msgr2=0x7fac7403ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fac7403c8a0 0x7fac7403ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fac80003980 tx=0x7fac80006e10 comp rx=0 tx=0).stop 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 msgr2=0x7fac8c19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c19b740 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac7c032ce0 tx=0x7fac7c032d10 comp rx=0 tx=0).stop 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 shutdown_connections 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fac7403c8a0 0x7fac7403ec90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:6800/986599852,v1:172.21.15.45:6801/986599852] conn(0x7fac74038f70 0x7fac7403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 --2- 172.21.15.45:0/1542715943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac8c1081e0 0x7fac8c19b740 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 >> 172.21.15.45:0/1542715943 conn(0x7fac8c101da0 msgr2=0x7fac8c1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 shutdown_connections 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:25.059+0000 7fac91832640 1 -- 172.21.15.45:0/1542715943 wait complete. 2024-09-10T09:54:25.619 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-09-10T09:54:25.620 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-09-10T09:54:26.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:25 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:26.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:25 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:26.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:25 smithi045 ceph-mon[31114]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T09:54:26.249 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:25 smithi045 ceph-mon[31114]: mgrmap e7: x(active, since 1.00528s) 2024-09-10T09:54:26.250 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:25 smithi045 ceph-mon[31114]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T09:54:26.853 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:26.506+0000 7fcbf0d22640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:09:54:26] ENGINE Error in HTTPServer.serve 2024-09-10T09:54:26.854 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: Traceback (most recent call last): 2024-09-10T09:54:26.854 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:54:26.854 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._connections.run(self.expiration_interval) 2024-09-10T09:54:26.854 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:54:26.854 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._run(expiration_interval) 2024-09-10T09:54:26.854 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:54:26.855 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:54:26.855 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:54:26.855 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:54:26.855 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:54:26.855 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: s = self.context.wrap_socket( 2024-09-10T09:54:26.855 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:54:26.855 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: return self.sslsocket_class._create( 2024-09-10T09:54:26.856 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:54:26.856 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self.do_handshake() 2024-09-10T09:54:26.856 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:54:26.856 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._sslobj.do_handshake() 2024-09-10T09:54:26.856 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:54:26.856 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:26.866 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.028+0000 7f5e0cd61640 1 Processor -- start 2024-09-10T09:54:26.866 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.028+0000 7f5e0cd61640 1 -- start start 2024-09-10T09:54:26.866 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.029+0000 7f5e0cd61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e08106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.029+0000 7f5e0cd61640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e08107000 con 0x7f5e08106630 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.029+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e08106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.029+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e08106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57264/0 (socket says 172.21.15.45:57264) 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.029+0000 7f5e0659c640 1 -- 172.21.15.45:0/2544961514 learned_addr learned my addr 172.21.15.45:0/2544961514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.029+0000 7f5e0659c640 1 -- 172.21.15.45:0/2544961514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e08107830 con 0x7f5e08106630 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.030+0000 7f5e0659c640 1 --2- 172.21.15.45:0/2544961514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e08106a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009820 tx=0x7f5df802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9e7095176aa5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.030+0000 7f5e05d9b640 1 -- 172.21.15.45:0/2544961514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5df8037070 con 0x7f5e08106630 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.030+0000 7f5e05d9b640 1 -- 172.21.15.45:0/2544961514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5df8032900 con 0x7f5e08106630 2024-09-10T09:54:26.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.030+0000 7f5e05d9b640 1 -- 172.21.15.45:0/2544961514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5df8030850 con 0x7f5e08106630 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.031+0000 7f5e0cd61640 1 -- 172.21.15.45:0/2544961514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 msgr2=0x7f5e08106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.031+0000 7f5e0cd61640 1 --2- 172.21.15.45:0/2544961514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e08106a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009820 tx=0x7f5df802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.031+0000 7f5e0cd61640 1 -- 172.21.15.45:0/2544961514 shutdown_connections 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.031+0000 7f5e0cd61640 1 --2- 172.21.15.45:0/2544961514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e08106a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.031+0000 7f5e0cd61640 1 -- 172.21.15.45:0/2544961514 >> 172.21.15.45:0/2544961514 conn(0x7f5e08101e20 msgr2=0x7f5e08104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.031+0000 7f5e0cd61640 1 -- 172.21.15.45:0/2544961514 shutdown_connections 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.031+0000 7f5e0cd61640 1 -- 172.21.15.45:0/2544961514 wait complete. 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.032+0000 7f5e0cd61640 1 Processor -- start 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.032+0000 7f5e0cd61640 1 -- start start 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.032+0000 7f5e0cd61640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e0819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.032+0000 7f5e0cd61640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0810abe0 con 0x7f5e08106630 2024-09-10T09:54:26.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.032+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e0819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5e0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e0819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57270/0 (socket says 172.21.15.45:57270) 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5e0659c640 1 -- 172.21.15.45:0/962654693 learned_addr learned my addr 172.21.15.45:0/962654693 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5e0659c640 1 -- 172.21.15.45:0/962654693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0819bd10 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5e0659c640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e0819b7d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009940 tx=0x7f5df80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5dff7fe640 1 -- 172.21.15.45:0/962654693 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5df8042070 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0819bfa0 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5dff7fe640 1 -- 172.21.15.45:0/962654693 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5df8032be0 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5dff7fe640 1 -- 172.21.15.45:0/962654693 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5df8037040 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.033+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0819ec90 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.034+0000 7f5dff7fe640 1 -- 172.21.15.45:0/962654693 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f5df8045070 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.034+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0810ad60 con 0x7f5e08106630 2024-09-10T09:54:26.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.035+0000 7f5dff7fe640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f5df4038e60 0x7f5df403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.035+0000 7f5dff7fe640 1 -- 172.21.15.45:0/962654693 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5df806bd10 con 0x7f5e08106630 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.035+0000 7f5dfffff640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f5df4038e60 0x7f5df403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.035+0000 7f5dfffff640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f5df4038e60 0x7f5df403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5df00098c0 tx=0x7f5df0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.040+0000 7f5dff7fe640 1 -- 172.21.15.45:0/962654693 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5df804b170 con 0x7f5e08106630 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.231+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f5e081030f0 con 0x7f5df4038e60 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.236+0000 7f5dff7fe640 1 -- 172.21.15.45:0/962654693 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5e081030f0 con 0x7f5df4038e60 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f5df4038e60 msgr2=0x7f5df403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f5df4038e60 0x7f5df403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5df00098c0 tx=0x7f5df0006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 msgr2=0x7f5e0819b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e0819b7d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009940 tx=0x7f5df80326e0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 shutdown_connections 2024-09-10T09:54:26.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f5df4038e60 0x7f5df403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 --2- 172.21.15.45:0/962654693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e08106630 0x7f5e0819b7d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 >> 172.21.15.45:0/962654693 conn(0x7f5e08101e20 msgr2=0x7f5e081029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 shutdown_connections 2024-09-10T09:54:26.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:26.238+0000 7f5e0cd61640 1 -- 172.21.15.45:0/962654693 wait complete. 2024-09-10T09:54:27.431 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:26] ENGINE Bus STARTING 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:26] ENGINE Serving on http://172.21.15.45:8765 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:26] ENGINE Serving on https://172.21.15.45:7150 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:26] ENGINE Bus STARTED 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:26] ENGINE Error in HTTPServer.serve 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: Traceback (most recent call last): 2024-09-10T09:54:27.432 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._connections.run(self.expiration_interval) 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._run(expiration_interval) 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: s = self.context.wrap_socket( 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: return self.sslsocket_class._create( 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:54:27.433 INFO:journalctl@ceph.mon.a.smithi045.stdout: self.do_handshake() 2024-09-10T09:54:27.434 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:54:27.434 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._sslobj.do_handshake() 2024-09-10T09:54:27.434 INFO:journalctl@ceph.mon.a.smithi045.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:54:27.434 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:27 smithi045 ceph-mon[31114]: mgrmap e8: x(active, since 2s) 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.243+0000 7f6493e43640 1 Processor -- start 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.243+0000 7f6493e43640 1 -- start start 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.244+0000 7f6493e43640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c10a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.244+0000 7f6493e43640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f648c07be90 con 0x7f648c1081a0 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.244+0000 7f6491bdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c10a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.244+0000 7f6491bdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c10a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57286/0 (socket says 172.21.15.45:57286) 2024-09-10T09:54:27.992 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.244+0000 7f6491bdf640 1 -- 172.21.15.45:0/2155993044 learned_addr learned my addr 172.21.15.45:0/2155993044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.244+0000 7f6491bdf640 1 -- 172.21.15.45:0/2155993044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c10aad0 con 0x7f648c1081a0 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6491bdf640 1 --2- 172.21.15.45:0/2155993044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c10a590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6488009820 tx=0x7f648802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fab1a9ef616da29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6490bdd640 1 -- 172.21.15.45:0/2155993044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6488037070 con 0x7f648c1081a0 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6490bdd640 1 -- 172.21.15.45:0/2155993044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6488032900 con 0x7f648c1081a0 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6490bdd640 1 -- 172.21.15.45:0/2155993044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6488030850 con 0x7f648c1081a0 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6493e43640 1 -- 172.21.15.45:0/2155993044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 msgr2=0x7f648c10a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6493e43640 1 --2- 172.21.15.45:0/2155993044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c10a590 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6488009820 tx=0x7f648802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6493e43640 1 -- 172.21.15.45:0/2155993044 shutdown_connections 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6493e43640 1 --2- 172.21.15.45:0/2155993044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c10a590 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6493e43640 1 -- 172.21.15.45:0/2155993044 >> 172.21.15.45:0/2155993044 conn(0x7f648c101e20 msgr2=0x7f648c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6493e43640 1 -- 172.21.15.45:0/2155993044 shutdown_connections 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.245+0000 7f6493e43640 1 -- 172.21.15.45:0/2155993044 wait complete. 2024-09-10T09:54:27.993 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.246+0000 7f6493e43640 1 Processor -- start 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.246+0000 7f6493e43640 1 -- start start 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.246+0000 7f6493e43640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.246+0000 7f6493e43640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f648c10bb70 con 0x7f648c1081a0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.246+0000 7f6491bdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.246+0000 7f6491bdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57302/0 (socket says 172.21.15.45:57302) 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.246+0000 7f6491bdf640 1 -- 172.21.15.45:0/323539476 learned_addr learned my addr 172.21.15.45:0/323539476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.247+0000 7f6491bdf640 1 -- 172.21.15.45:0/323539476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648c19df50 con 0x7f648c1081a0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.247+0000 7f6491bdf640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c19da10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64880097f0 tx=0x7f6488030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.247+0000 7f6482ffd640 1 -- 172.21.15.45:0/323539476 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6488042070 con 0x7f648c1081a0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.247+0000 7f6482ffd640 1 -- 172.21.15.45:0/323539476 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6488032ca0 con 0x7f648c1081a0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.247+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648c19e1e0 con 0x7f648c1081a0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.247+0000 7f6482ffd640 1 -- 172.21.15.45:0/323539476 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6488037040 con 0x7f648c1081a0 2024-09-10T09:54:27.994 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.247+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648c1a0ed0 con 0x7f648c1081a0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.248+0000 7f6482ffd640 1 -- 172.21.15.45:0/323539476 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6488045070 con 0x7f648c1081a0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.248+0000 7f6482ffd640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f6478038fc0 0x7f647803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.248+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648c108600 con 0x7f648c1081a0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.248+0000 7f6482ffd640 1 -- 172.21.15.45:0/323539476 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f648806bef0 con 0x7f648c1081a0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.248+0000 7f64913de640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f6478038fc0 0x7f647803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.249+0000 7f64913de640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f6478038fc0 0x7f647803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f647c0098c0 tx=0x7f647c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.253+0000 7f6482ffd640 1 -- 172.21.15.45:0/323539476 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f648804b170 con 0x7f648c1081a0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.430+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f648c1030a0 con 0x7f6478038fc0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.431+0000 7f6482ffd640 1 -- 172.21.15.45:0/323539476 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f648c1030a0 con 0x7f6478038fc0 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f6478038fc0 msgr2=0x7f647803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f6478038fc0 0x7f647803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f647c0098c0 tx=0x7f647c006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 msgr2=0x7f648c19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:27.995 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c19da10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64880097f0 tx=0x7f6488030ec0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 shutdown_connections 2024-09-10T09:54:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f6478038fc0 0x7f647803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 --2- 172.21.15.45:0/323539476 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f648c1081a0 0x7f648c19da10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 >> 172.21.15.45:0/323539476 conn(0x7f648c101e20 msgr2=0x7f648c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 shutdown_connections 2024-09-10T09:54:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:27.433+0000 7f6493e43640 1 -- 172.21.15.45:0/323539476 wait complete. 2024-09-10T09:54:27.996 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-09-10T09:54:28.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-mon[31114]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:28.676 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: Generating public/private rsa key pair. 2024-09-10T09:54:28.676 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: Your identification has been saved in /tmp/tmp4ieh5afn/key 2024-09-10T09:54:28.676 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: Your public key has been saved in /tmp/tmp4ieh5afn/key.pub 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: The key fingerprint is: 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: SHA256:A/UvHxYecY23U6f1QV/J4gJwnnnpdANsOPH8BTh8jAA ceph-729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: The key's randomart image is: 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: +---[RSA 3072]----+ 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | E++*.=.++o| 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | .o==Bo*o+O| 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | . +=**o.=B| 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | . ++o++o.| 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | S ..*. .| 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | . + . | 2024-09-10T09:54:28.677 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | . | 2024-09-10T09:54:28.678 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | | 2024-09-10T09:54:28.678 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: | | 2024-09-10T09:54:28.678 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: +----[SHA256]-----+ 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.417+0000 7ff362318640 1 Processor -- start 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.417+0000 7ff362318640 1 -- start start 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.417+0000 7ff362318640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.417+0000 7ff362318640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c07c8b0 con 0x7ff35c07de80 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.417+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.417+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57318/0 (socket says 172.21.15.45:57318) 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.417+0000 7ff35bfff640 1 -- 172.21.15.45:0/2112385737 learned_addr learned my addr 172.21.15.45:0/2112385737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:29.264 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.418+0000 7ff35bfff640 1 -- 172.21.15.45:0/2112385737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c07ca30 con 0x7ff35c07de80 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.418+0000 7ff35bfff640 1 --2- 172.21.15.45:0/2112385737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c07c2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009820 tx=0x7ff34c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a906927338d9e8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.418+0000 7ff35affd640 1 -- 172.21.15.45:0/2112385737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff34c037070 con 0x7ff35c07de80 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.418+0000 7ff35affd640 1 -- 172.21.15.45:0/2112385737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff34c032900 con 0x7ff35c07de80 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.418+0000 7ff35affd640 1 -- 172.21.15.45:0/2112385737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff34c030850 con 0x7ff35c07de80 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.419+0000 7ff362318640 1 -- 172.21.15.45:0/2112385737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 msgr2=0x7ff35c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.419+0000 7ff362318640 1 --2- 172.21.15.45:0/2112385737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c07c2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009820 tx=0x7ff34c02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.419+0000 7ff362318640 1 -- 172.21.15.45:0/2112385737 shutdown_connections 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.419+0000 7ff362318640 1 --2- 172.21.15.45:0/2112385737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c07c2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.419+0000 7ff362318640 1 -- 172.21.15.45:0/2112385737 >> 172.21.15.45:0/2112385737 conn(0x7ff35c102e80 msgr2=0x7ff35c1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.419+0000 7ff362318640 1 -- 172.21.15.45:0/2112385737 shutdown_connections 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.419+0000 7ff362318640 1 -- 172.21.15.45:0/2112385737 wait complete. 2024-09-10T09:54:29.265 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff362318640 1 Processor -- start 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff362318640 1 -- start start 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff362318640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff362318640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c10dd60 con 0x7ff35c07de80 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff35bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57330/0 (socket says 172.21.15.45:57330) 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff35bfff640 1 -- 172.21.15.45:0/1266842360 learned_addr learned my addr 172.21.15.45:0/1266842360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.420+0000 7ff35bfff640 1 -- 172.21.15.45:0/1266842360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c19de40 con 0x7ff35c07de80 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.421+0000 7ff35bfff640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c19d900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0097f0 tx=0x7ff34c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.421+0000 7ff3597fa640 1 -- 172.21.15.45:0/1266842360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff34c037040 con 0x7ff35c07de80 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.421+0000 7ff3597fa640 1 -- 172.21.15.45:0/1266842360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff34c040070 con 0x7ff35c07de80 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.421+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c19e0d0 con 0x7ff35c07de80 2024-09-10T09:54:29.266 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.421+0000 7ff3597fa640 1 -- 172.21.15.45:0/1266842360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff34c03a390 con 0x7ff35c07de80 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.421+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c1a0dc0 con 0x7ff35c07de80 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.422+0000 7ff3597fa640 1 -- 172.21.15.45:0/1266842360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff34c03b430 con 0x7ff35c07de80 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.422+0000 7ff3597fa640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7ff344038f70 0x7ff34403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.422+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c07c360 con 0x7ff35c07de80 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.422+0000 7ff3597fa640 1 -- 172.21.15.45:0/1266842360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff34c06c4a0 con 0x7ff35c07de80 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.422+0000 7ff35b7fe640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7ff344038f70 0x7ff34403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.422+0000 7ff35b7fe640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7ff344038f70 0x7ff34403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3500098c0 tx=0x7ff350006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.427+0000 7ff3597fa640 1 -- 172.21.15.45:0/1266842360 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff34c03a5e0 con 0x7ff35c07de80 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.603+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7ff35c104150 con 0x7ff344038f70 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.663+0000 7ff3597fa640 1 -- 172.21.15.45:0/1266842360 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff35c104150 con 0x7ff344038f70 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7ff344038f70 msgr2=0x7ff34403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7ff344038f70 0x7ff34403b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff3500098c0 tx=0x7ff350006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.267 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 msgr2=0x7ff35c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c19d900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0097f0 tx=0x7ff34c030d80 comp rx=0 tx=0).stop 2024-09-10T09:54:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 shutdown_connections 2024-09-10T09:54:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7ff344038f70 0x7ff34403b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 --2- 172.21.15.45:0/1266842360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff35c07de80 0x7ff35c19d900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 >> 172.21.15.45:0/1266842360 conn(0x7ff35c102e80 msgr2=0x7ff35c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 shutdown_connections 2024-09-10T09:54:29.268 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:28.665+0000 7ff362318640 1 -- 172.21.15.45:0/1266842360 wait complete. 2024-09-10T09:54:29.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:29 smithi045 ceph-mon[31114]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:29.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:29 smithi045 ceph-mon[31114]: Generating ssh key... 2024-09-10T09:54:29.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:29 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:29.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:29 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:30.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8IApLOVeOHtvaZle40hZ/i0UuxQbbFTC2FDoS2YjSGl5ARK5OjJMF8fQapT/Y9ffgfu/Aymmo3b77dv74EJ2EG4jIxeoAWmuJHs8ILxOU89jE58WBb1BpYx83q4IUwdiVotUAIi6JSOTG67NWOkerc6NZ2kMe9UbLC/KILR07SCYRV5gInVzlcNlC2Pb7v5g46S5OCzNLivVD1pEK6XJSiDk8pLFd7z9jL2sXn/OFiQQKWZ3b0wIR4+qbJrezLQJYBE0RGAT28yuWxVZgkMIKDtl/2UCvk06SZzwj+JqGni491gKlbU1qyJvMattCC9/O11tcd/NzBb5Ugr+Kg3PoH+drpT0C2Y7DJwovaxTtIMBkXCd1wq4n1J4EVN/RVpkelhTf6N2sf9Rwcb1I7740xh4x//srh0yZpz4/sJP0XzWmLAks+e1EHWhTDsWG2Un+Dwtaa71KbxKrQSHKYD8TSJmxXrSCJ6KTRP0pGfLa7xDRbQM97LOEeF89Dshle1c= ceph-729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:30.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.704+0000 7fe8372ec640 1 Processor -- start 2024-09-10T09:54:30.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.704+0000 7fe8372ec640 1 -- start start 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.704+0000 7fe8372ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.704+0000 7fe8372ec640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe83007be30 con 0x7fe8301081e0 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.704+0000 7fe835088640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.704+0000 7fe835088640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57344/0 (socket says 172.21.15.45:57344) 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.704+0000 7fe835088640 1 -- 172.21.15.45:0/1848769893 learned_addr learned my addr 172.21.15.45:0/1848769893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.705+0000 7fe835088640 1 -- 172.21.15.45:0/1848769893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83010ab10 con 0x7fe8301081e0 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.705+0000 7fe835088640 1 --2- 172.21.15.45:0/1848769893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83010a5d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009820 tx=0x7fe82c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b6066757e62ce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe827fff640 1 -- 172.21.15.45:0/1848769893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe82c037070 con 0x7fe8301081e0 2024-09-10T09:54:30.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe827fff640 1 -- 172.21.15.45:0/1848769893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe82c032900 con 0x7fe8301081e0 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe827fff640 1 -- 172.21.15.45:0/1848769893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe82c030850 con 0x7fe8301081e0 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe8372ec640 1 -- 172.21.15.45:0/1848769893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 msgr2=0x7fe83010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe8372ec640 1 --2- 172.21.15.45:0/1848769893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83010a5d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009820 tx=0x7fe82c02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe8372ec640 1 -- 172.21.15.45:0/1848769893 shutdown_connections 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe8372ec640 1 --2- 172.21.15.45:0/1848769893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83010a5d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe8372ec640 1 -- 172.21.15.45:0/1848769893 >> 172.21.15.45:0/1848769893 conn(0x7fe830101da0 msgr2=0x7fe8301041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe8372ec640 1 -- 172.21.15.45:0/1848769893 shutdown_connections 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.706+0000 7fe8372ec640 1 -- 172.21.15.45:0/1848769893 wait complete. 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.707+0000 7fe8372ec640 1 Processor -- start 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.707+0000 7fe8372ec640 1 -- start start 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.707+0000 7fe8372ec640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.707+0000 7fe8372ec640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe83010bb50 con 0x7fe8301081e0 2024-09-10T09:54:30.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.707+0000 7fe835088640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.707+0000 7fe835088640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57348/0 (socket says 172.21.15.45:57348) 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe835088640 1 -- 172.21.15.45:0/185258260 learned_addr learned my addr 172.21.15.45:0/185258260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe835088640 1 -- 172.21.15.45:0/185258260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83019dee0 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe835088640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83019d9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe82c032ce0 tx=0x7fe82c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe8267fc640 1 -- 172.21.15.45:0/185258260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe82c042070 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe8267fc640 1 -- 172.21.15.45:0/185258260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe82c0308c0 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83019e170 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe8267fc640 1 -- 172.21.15.45:0/185258260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe82c037040 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.708+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8301a0e60 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.709+0000 7fe8267fc640 1 -- 172.21.15.45:0/185258260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe82c045070 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.709+0000 7fe8267fc640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fe81c038fc0 0x7fe81c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.709+0000 7fe8267fc640 1 -- 172.21.15.45:0/185258260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe82c06c040 con 0x7fe8301081e0 2024-09-10T09:54:30.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.709+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe830108640 con 0x7fe8301081e0 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.710+0000 7fe834887640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fe81c038fc0 0x7fe81c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.710+0000 7fe834887640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fe81c038fc0 0x7fe81c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8200098c0 tx=0x7fe820006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.714+0000 7fe8267fc640 1 -- 172.21.15.45:0/185258260 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe82c04b170 con 0x7fe8301081e0 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.892+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe830102fe0 con 0x7fe81c038fc0 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.893+0000 7fe8267fc640 1 -- 172.21.15.45:0/185258260 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe830102fe0 con 0x7fe81c038fc0 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.895+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fe81c038fc0 msgr2=0x7fe81c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.895+0000 7fe8372ec640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fe81c038fc0 0x7fe81c03b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe8200098c0 tx=0x7fe820006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.895+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 msgr2=0x7fe83019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.895+0000 7fe8372ec640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83019d9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe82c032ce0 tx=0x7fe82c032d10 comp rx=0 tx=0).stop 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.896+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 shutdown_connections 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.896+0000 7fe8372ec640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fe81c038fc0 0x7fe81c03b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.896+0000 7fe8372ec640 1 --2- 172.21.15.45:0/185258260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8301081e0 0x7fe83019d9a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.896+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 >> 172.21.15.45:0/185258260 conn(0x7fe830101da0 msgr2=0x7fe8301028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:30.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.896+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 shutdown_connections 2024-09-10T09:54:30.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:29.896+0000 7fe8372ec640 1 -- 172.21.15.45:0/185258260 wait complete. 2024-09-10T09:54:30.458 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T09:54:30.458 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T09:54:30.458 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-09-10T09:54:30.740 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:30 smithi045 ceph-mon[31114]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:32.035 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:31 smithi045 ceph-mon[31114]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:33.244 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:33 smithi045 ceph-mon[31114]: Deploying cephadm binary to smithi045 2024-09-10T09:54:35.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-09-10T09:54:35.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.825+0000 7f82c11bf640 1 Processor -- start 2024-09-10T09:54:35.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.825+0000 7f82c11bf640 1 -- start start 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.826+0000 7f82c11bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc10a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.826+0000 7f82c11bf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc07be30 con 0x7f82bc1081e0 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.826+0000 7f82bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc10a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.826+0000 7f82bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc10a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57360/0 (socket says 172.21.15.45:57360) 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.826+0000 7f82bad9d640 1 -- 172.21.15.45:0/1898442992 learned_addr learned my addr 172.21.15.45:0/1898442992 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.826+0000 7f82bad9d640 1 -- 172.21.15.45:0/1898442992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc10ab10 con 0x7f82bc1081e0 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.827+0000 7f82bad9d640 1 --2- 172.21.15.45:0/1898442992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc10a5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009820 tx=0x7f82ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82455907249ca231 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.827+0000 7f82b9d9b640 1 -- 172.21.15.45:0/1898442992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ac037070 con 0x7f82bc1081e0 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.827+0000 7f82b9d9b640 1 -- 172.21.15.45:0/1898442992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f82ac032900 con 0x7f82bc1081e0 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.827+0000 7f82b9d9b640 1 -- 172.21.15.45:0/1898442992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ac030850 con 0x7f82bc1081e0 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 -- 172.21.15.45:0/1898442992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 msgr2=0x7f82bc10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 --2- 172.21.15.45:0/1898442992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc10a5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f82ac009820 tx=0x7f82ac02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 -- 172.21.15.45:0/1898442992 shutdown_connections 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 --2- 172.21.15.45:0/1898442992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc10a5d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 -- 172.21.15.45:0/1898442992 >> 172.21.15.45:0/1898442992 conn(0x7f82bc101da0 msgr2=0x7f82bc1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 -- 172.21.15.45:0/1898442992 shutdown_connections 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 -- 172.21.15.45:0/1898442992 wait complete. 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.828+0000 7f82c11bf640 1 Processor -- start 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82c11bf640 1 -- start start 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82c11bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82c11bf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc10bb50 con 0x7f82bc1081e0 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57374/0 (socket says 172.21.15.45:57374) 2024-09-10T09:54:35.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82bad9d640 1 -- 172.21.15.45:0/2802429776 learned_addr learned my addr 172.21.15.45:0/2802429776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82bad9d640 1 -- 172.21.15.45:0/2802429776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc19bc90 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.829+0000 7f82bad9d640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc19b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82ac032ce0 tx=0x7f82ac032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.830+0000 7f82a3fff640 1 -- 172.21.15.45:0/2802429776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ac042070 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.830+0000 7f82a3fff640 1 -- 172.21.15.45:0/2802429776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f82ac0308c0 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.830+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc19bf20 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.830+0000 7f82a3fff640 1 -- 172.21.15.45:0/2802429776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ac037040 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.830+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc19ebc0 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.831+0000 7f82a3fff640 1 -- 172.21.15.45:0/2802429776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f82ac045070 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.831+0000 7f82a3fff640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f82a4038fc0 0x7f82a403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.831+0000 7f82a3fff640 1 -- 172.21.15.45:0/2802429776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f82ac06c160 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.831+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc108640 con 0x7f82bc1081e0 2024-09-10T09:54:35.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.831+0000 7f82ba59c640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f82a4038fc0 0x7f82a403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.831+0000 7f82ba59c640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f82a4038fc0 0x7f82a403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82b00098c0 tx=0x7f82b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:30.836+0000 7f82a3fff640 1 -- 172.21.15.45:0/2802429776 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82ac045360 con 0x7f82bc1081e0 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:31.014+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) -- 0x7f82bc103060 con 0x7f82a4038fc0 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.883+0000 7f82a3fff640 1 -- 172.21.15.45:0/2802429776 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f82bc103060 con 0x7f82a4038fc0 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f82a4038fc0 msgr2=0x7f82a403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f82a4038fc0 0x7f82a403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f82b00098c0 tx=0x7f82b0006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 msgr2=0x7f82bc19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc19b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82ac032ce0 tx=0x7f82ac032d10 comp rx=0 tx=0).stop 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 shutdown_connections 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f82a4038fc0 0x7f82a403b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.457 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 --2- 172.21.15.45:0/2802429776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82bc1081e0 0x7f82bc19b750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:35.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 >> 172.21.15.45:0/2802429776 conn(0x7f82bc101da0 msgr2=0x7f82bc102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:35.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 shutdown_connections 2024-09-10T09:54:35.458 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:34.886+0000 7f82c11bf640 1 -- 172.21.15.45:0/2802429776 wait complete. 2024-09-10T09:54:35.458 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-09-10T09:54:36.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:35 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:36.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:35 smithi045 ceph-mon[31114]: Added host smithi045 2024-09-10T09:54:36.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:35 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T09:54:36.820 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.825+0000 7fa855ff4640 1 Processor -- start 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.825+0000 7fa855ff4640 1 -- start start 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.825+0000 7fa855ff4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa850107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.825+0000 7fa855ff4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa8501080c0 con 0x7fa8501076f0 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.825+0000 7fa84f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa850107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.825+0000 7fa84f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa850107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45492/0 (socket says 172.21.15.45:45492) 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.825+0000 7fa84f7fe640 1 -- 172.21.15.45:0/2653615524 learned_addr learned my addr 172.21.15.45:0/2653615524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.826+0000 7fa84f7fe640 1 -- 172.21.15.45:0/2653615524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8501088f0 con 0x7fa8501076f0 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.826+0000 7fa84f7fe640 1 --2- 172.21.15.45:0/2653615524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa850107af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa840009820 tx=0x7fa84002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9ec7c59023e786 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.826+0000 7fa84e7fc640 1 -- 172.21.15.45:0/2653615524 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa840037070 con 0x7fa8501076f0 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.826+0000 7fa84e7fc640 1 -- 172.21.15.45:0/2653615524 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa840032900 con 0x7fa8501076f0 2024-09-10T09:54:36.821 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.826+0000 7fa84e7fc640 1 -- 172.21.15.45:0/2653615524 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa840030850 con 0x7fa8501076f0 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.827+0000 7fa855ff4640 1 -- 172.21.15.45:0/2653615524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 msgr2=0x7fa850107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.827+0000 7fa855ff4640 1 --2- 172.21.15.45:0/2653615524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa850107af0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa840009820 tx=0x7fa84002a270 comp rx=0 tx=0).stop 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.827+0000 7fa855ff4640 1 -- 172.21.15.45:0/2653615524 shutdown_connections 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.827+0000 7fa855ff4640 1 --2- 172.21.15.45:0/2653615524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa850107af0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.827+0000 7fa855ff4640 1 -- 172.21.15.45:0/2653615524 >> 172.21.15.45:0/2653615524 conn(0x7fa850102e80 msgr2=0x7fa8501052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.827+0000 7fa855ff4640 1 -- 172.21.15.45:0/2653615524 shutdown_connections 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.827+0000 7fa855ff4640 1 -- 172.21.15.45:0/2653615524 wait complete. 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa855ff4640 1 Processor -- start 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa855ff4640 1 -- start start 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa855ff4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa85019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa855ff4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa85010bca0 con 0x7fa8501076f0 2024-09-10T09:54:36.822 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa84f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa85019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa84f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa85019b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45498/0 (socket says 172.21.15.45:45498) 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa84f7fe640 1 -- 172.21.15.45:0/1512955223 learned_addr learned my addr 172.21.15.45:0/1512955223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa84f7fe640 1 -- 172.21.15.45:0/1512955223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85019bc80 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.828+0000 7fa84f7fe640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa85019b740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa840009940 tx=0x7fa840032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.829+0000 7fa84cff9640 1 -- 172.21.15.45:0/1512955223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa840042070 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.829+0000 7fa84cff9640 1 -- 172.21.15.45:0/1512955223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8400308c0 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.829+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85019bf10 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.829+0000 7fa84cff9640 1 -- 172.21.15.45:0/1512955223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa840037040 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.829+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85019ec00 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.830+0000 7fa84cff9640 1 -- 172.21.15.45:0/1512955223 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa840045070 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.830+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85010be20 con 0x7fa8501076f0 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.830+0000 7fa84cff9640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fa838038f70 0x7fa83803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.823 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.830+0000 7fa84cff9640 1 -- 172.21.15.45:0/1512955223 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa84006c060 con 0x7fa8501076f0 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.830+0000 7fa84effd640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fa838038f70 0x7fa83803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.830+0000 7fa84effd640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fa838038f70 0x7fa83803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa8440098c0 tx=0x7fa844006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:35.835+0000 7fa84cff9640 1 -- 172.21.15.45:0/1512955223 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa84003cd90 con 0x7fa8501076f0 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.022+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa850104130 con 0x7fa838038f70 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.027+0000 7fa84cff9640 1 -- 172.21.15.45:0/1512955223 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa850104130 con 0x7fa838038f70 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fa838038f70 msgr2=0x7fa83803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fa838038f70 0x7fa83803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa8440098c0 tx=0x7fa844006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 msgr2=0x7fa85019b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa85019b740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa840009940 tx=0x7fa840032ce0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 shutdown_connections 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fa838038f70 0x7fa83803b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.824 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 --2- 172.21.15.45:0/1512955223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8501076f0 0x7fa85019b740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 >> 172.21.15.45:0/1512955223 conn(0x7fa850102e80 msgr2=0x7fa850103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 shutdown_connections 2024-09-10T09:54:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:36.029+0000 7fa855ff4640 1 -- 172.21.15.45:0/1512955223 wait complete. 2024-09-10T09:54:36.825 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-09-10T09:54:37.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:37 smithi045 ceph-mon[31114]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:37.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:37 smithi045 ceph-mon[31114]: Saving service mon spec with placement count:5 2024-09-10T09:54:37.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:37 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:38.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:38 smithi045 ceph-mon[31114]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:38.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:38 smithi045 ceph-mon[31114]: Saving service mgr spec with placement count:2 2024-09-10T09:54:38.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:38 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:38.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:38 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.216+0000 7f0b1469c640 1 Processor -- start 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.216+0000 7f0b1469c640 1 -- start start 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.217+0000 7f0b1469c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.217+0000 7f0b1469c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c07be90 con 0x7f0b0c108180 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.217+0000 7f0b12438640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.217+0000 7f0b12438640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45512/0 (socket says 172.21.15.45:45512) 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.217+0000 7f0b12438640 1 -- 172.21.15.45:0/2675647373 learned_addr learned my addr 172.21.15.45:0/2675647373 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:38.780 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.217+0000 7f0b12438640 1 -- 172.21.15.45:0/2675647373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c10aab0 con 0x7f0b0c108180 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.218+0000 7f0b12438640 1 --2- 172.21.15.45:0/2675647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c10a570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009820 tx=0x7f0b0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c73a1f0a580345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.218+0000 7f0b11436640 1 -- 172.21.15.45:0/2675647373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b08037070 con 0x7f0b0c108180 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.218+0000 7f0b11436640 1 -- 172.21.15.45:0/2675647373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b08032900 con 0x7f0b0c108180 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.218+0000 7f0b11436640 1 -- 172.21.15.45:0/2675647373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b08030850 con 0x7f0b0c108180 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.219+0000 7f0b1469c640 1 -- 172.21.15.45:0/2675647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 msgr2=0x7f0b0c10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.219+0000 7f0b1469c640 1 --2- 172.21.15.45:0/2675647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c10a570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009820 tx=0x7f0b0802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.219+0000 7f0b1469c640 1 -- 172.21.15.45:0/2675647373 shutdown_connections 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.219+0000 7f0b1469c640 1 --2- 172.21.15.45:0/2675647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c10a570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.219+0000 7f0b1469c640 1 -- 172.21.15.45:0/2675647373 >> 172.21.15.45:0/2675647373 conn(0x7f0b0c101da0 msgr2=0x7f0b0c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.219+0000 7f0b1469c640 1 -- 172.21.15.45:0/2675647373 shutdown_connections 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.219+0000 7f0b1469c640 1 -- 172.21.15.45:0/2675647373 wait complete. 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b1469c640 1 Processor -- start 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b1469c640 1 -- start start 2024-09-10T09:54:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b1469c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b1469c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c10bb50 con 0x7f0b0c108180 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b12438640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b12438640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45520/0 (socket says 172.21.15.45:45520) 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b12438640 1 -- 172.21.15.45:0/95135047 learned_addr learned my addr 172.21.15.45:0/95135047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b12438640 1 -- 172.21.15.45:0/95135047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c19bc50 con 0x7f0b0c108180 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.220+0000 7f0b12438640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c19b710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b080097f0 tx=0x7f0b08030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.221+0000 7f0b037fe640 1 -- 172.21.15.45:0/95135047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b08037040 con 0x7f0b0c108180 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.221+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c19bee0 con 0x7f0b0c108180 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.221+0000 7f0b037fe640 1 -- 172.21.15.45:0/95135047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b08040070 con 0x7f0b0c108180 2024-09-10T09:54:38.782 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.221+0000 7f0b037fe640 1 -- 172.21.15.45:0/95135047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b0803a390 con 0x7f0b0c108180 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.221+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c19ebd0 con 0x7f0b0c108180 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.222+0000 7f0b037fe640 1 -- 172.21.15.45:0/95135047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0b0803a650 con 0x7f0b0c108180 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.222+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c1085e0 con 0x7f0b0c108180 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.222+0000 7f0b037fe640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0af8038f70 0x7f0af803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.222+0000 7f0b037fe640 1 -- 172.21.15.45:0/95135047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b0806c5f0 con 0x7f0b0c108180 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.222+0000 7f0b11c37640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0af8038f70 0x7f0af803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.222+0000 7f0b11c37640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0af8038f70 0x7f0af803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0098c0 tx=0x7f0afc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.227+0000 7f0b037fe640 1 -- 172.21.15.45:0/95135047 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b08042e30 con 0x7f0b0c108180 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.418+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0b0c1030b0 con 0x7f0af8038f70 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.422+0000 7f0b037fe640 1 -- 172.21.15.45:0/95135047 <== mgr.14118 v2:172.21.15.45:6800/1958743370 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0b0c1030b0 con 0x7f0af8038f70 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0af8038f70 msgr2=0x7f0af803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0af8038f70 0x7f0af803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0098c0 tx=0x7f0afc006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.783 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 msgr2=0x7f0b0c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c19b710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b080097f0 tx=0x7f0b08030d80 comp rx=0 tx=0).stop 2024-09-10T09:54:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 shutdown_connections 2024-09-10T09:54:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0af8038f70 0x7f0af803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 --2- 172.21.15.45:0/95135047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b0c108180 0x7f0b0c19b710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 >> 172.21.15.45:0/95135047 conn(0x7f0b0c101da0 msgr2=0x7f0b0c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 shutdown_connections 2024-09-10T09:54:38.784 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:37.425+0000 7f0b1469c640 1 -- 172.21.15.45:0/95135047 wait complete. 2024-09-10T09:54:40.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:39 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:40.038 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:39 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/682011770' entity='client.admin' 2024-09-10T09:54:40.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.224+0000 7f0e9053d640 1 Processor -- start 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.224+0000 7f0e9053d640 1 -- start start 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.224+0000 7f0e9053d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e88106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.224+0000 7f0e9053d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8807be10 con 0x7f0e88106010 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.225+0000 7f0e8e2d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e88106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.225+0000 7f0e8e2d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e88106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45524/0 (socket says 172.21.15.45:45524) 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.225+0000 7f0e8e2d9640 1 -- 172.21.15.45:0/1751314775 learned_addr learned my addr 172.21.15.45:0/1751314775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.225+0000 7f0e8e2d9640 1 -- 172.21.15.45:0/1751314775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e88106950 con 0x7f0e88106010 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.226+0000 7f0e8e2d9640 1 --2- 172.21.15.45:0/1751314775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e88106410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009e80 tx=0x7f0e7802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608b2b1b1f4c3a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.226+0000 7f0e8d2d7640 1 -- 172.21.15.45:0/1751314775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e78036070 con 0x7f0e88106010 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.226+0000 7f0e8d2d7640 1 -- 172.21.15.45:0/1751314775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e780318e0 con 0x7f0e88106010 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.226+0000 7f0e8d2d7640 1 -- 172.21.15.45:0/1751314775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e78007a00 con 0x7f0e88106010 2024-09-10T09:54:40.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.226+0000 7f0e9053d640 1 -- 172.21.15.45:0/1751314775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 msgr2=0x7f0e88106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.226+0000 7f0e9053d640 1 --2- 172.21.15.45:0/1751314775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e88106410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009e80 tx=0x7f0e7802a7a0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 -- 172.21.15.45:0/1751314775 shutdown_connections 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 --2- 172.21.15.45:0/1751314775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e88106410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 -- 172.21.15.45:0/1751314775 >> 172.21.15.45:0/1751314775 conn(0x7f0e88101e20 msgr2=0x7f0e88104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 -- 172.21.15.45:0/1751314775 shutdown_connections 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 -- 172.21.15.45:0/1751314775 wait complete. 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 Processor -- start 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 -- start start 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e8819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.227+0000 7f0e9053d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e88107a90 con 0x7f0e88106010 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e8e2d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e8819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e8e2d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e8819d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45540/0 (socket says 172.21.15.45:45540) 2024-09-10T09:54:40.047 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e8e2d9640 1 -- 172.21.15.45:0/682011770 learned_addr learned my addr 172.21.15.45:0/682011770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e8e2d9640 1 -- 172.21.15.45:0/682011770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8819dec0 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e8e2d9640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e8819d980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e7802acc0 tx=0x7f0e78031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e877fe640 1 -- 172.21.15.45:0/682011770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e78036050 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e877fe640 1 -- 172.21.15.45:0/682011770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e78037040 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8819e150 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e877fe640 1 -- 172.21.15.45:0/682011770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e7803d970 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.228+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e881a0e40 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.229+0000 7f0e877fe640 1 -- 172.21.15.45:0/682011770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0e78044020 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.229+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e88106410 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.229+0000 7f0e877fe640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0e70038fc0 0x7f0e7003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.229+0000 7f0e877fe640 1 -- 172.21.15.45:0/682011770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e78042020 con 0x7f0e88106010 2024-09-10T09:54:40.048 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.229+0000 7f0e8dad8640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0e70038fc0 0x7f0e7003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.230+0000 7f0e8dad8640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0e70038fc0 0x7f0e7003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0098c0 tx=0x7f0e7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.234+0000 7f0e877fe640 1 -- 172.21.15.45:0/682011770 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e7804b020 con 0x7f0e88106010 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.416+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f0e88196df0 con 0x7f0e88106010 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.419+0000 7f0e877fe640 1 -- 172.21.15.45:0/682011770 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f0e7803bc60 con 0x7f0e88106010 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.421+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0e70038fc0 msgr2=0x7f0e7003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.421+0000 7f0e9053d640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0e70038fc0 0x7f0e7003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0098c0 tx=0x7f0e7c006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.421+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 msgr2=0x7f0e8819d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.421+0000 7f0e9053d640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e8819d980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e7802acc0 tx=0x7f0e78031cc0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.422+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 shutdown_connections 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.422+0000 7f0e9053d640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f0e70038fc0 0x7f0e7003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.422+0000 7f0e9053d640 1 --2- 172.21.15.45:0/682011770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e88106010 0x7f0e8819d980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.049 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.422+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 >> 172.21.15.45:0/682011770 conn(0x7f0e88101e20 msgr2=0x7f0e88191c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.422+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 shutdown_connections 2024-09-10T09:54:40.050 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:39.422+0000 7f0e9053d640 1 -- 172.21.15.45:0/682011770 wait complete. 2024-09-10T09:54:41.872 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:41 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:41.873 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:41 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2584600667' entity='client.admin' 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.083+0000 7fbea2949640 1 Processor -- start 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.084+0000 7fbea2949640 1 -- start start 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.084+0000 7fbea2949640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c10a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.084+0000 7fbea2949640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c07be50 con 0x7fbe9c1081c0 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.084+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c10a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.084+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c10a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45544/0 (socket says 172.21.15.45:45544) 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.084+0000 7fbe9bfff640 1 -- 172.21.15.45:0/1391361409 learned_addr learned my addr 172.21.15.45:0/1391361409 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.084+0000 7fbe9bfff640 1 -- 172.21.15.45:0/1391361409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c10aaf0 con 0x7fbe9c1081c0 2024-09-10T09:54:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.085+0000 7fbe9bfff640 1 --2- 172.21.15.45:0/1391361409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c10a5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009820 tx=0x7fbe8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0f0dd5ba6ef1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.085+0000 7fbe9affd640 1 -- 172.21.15.45:0/1391361409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c037070 con 0x7fbe9c1081c0 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.085+0000 7fbe9affd640 1 -- 172.21.15.45:0/1391361409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c032920 con 0x7fbe9c1081c0 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.085+0000 7fbe9affd640 1 -- 172.21.15.45:0/1391361409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c030810 con 0x7fbe9c1081c0 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 -- 172.21.15.45:0/1391361409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 msgr2=0x7fbe9c10a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 --2- 172.21.15.45:0/1391361409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c10a5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009820 tx=0x7fbe8c02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 -- 172.21.15.45:0/1391361409 shutdown_connections 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 --2- 172.21.15.45:0/1391361409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c10a5b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 -- 172.21.15.45:0/1391361409 >> 172.21.15.45:0/1391361409 conn(0x7fbe9c101da0 msgr2=0x7fbe9c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 -- 172.21.15.45:0/1391361409 shutdown_connections 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 -- 172.21.15.45:0/1391361409 wait complete. 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 Processor -- start 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.086+0000 7fbea2949640 1 -- start start 2024-09-10T09:54:41.881 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbea2949640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbea2949640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c10bb70 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c19b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45552/0 (socket says 172.21.15.45:45552) 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe9bfff640 1 -- 172.21.15.45:0/2584600667 learned_addr learned my addr 172.21.15.45:0/2584600667 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe9bfff640 1 -- 172.21.15.45:0/2584600667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c19bc70 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe9bfff640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c19b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0097f0 tx=0x7fbe8c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe997fa640 1 -- 172.21.15.45:0/2584600667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c037040 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe997fa640 1 -- 172.21.15.45:0/2584600667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c030cc0 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c19bf00 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.087+0000 7fbe997fa640 1 -- 172.21.15.45:0/2584600667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c03a6b0 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.088+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c19ebf0 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.089+0000 7fbe997fa640 1 -- 172.21.15.45:0/2584600667 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbe8c03baf0 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.089+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c108620 con 0x7fbe9c1081c0 2024-09-10T09:54:41.882 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.089+0000 7fbe997fa640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fbe84038fc0 0x7fbe8403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.089+0000 7fbe997fa640 1 -- 172.21.15.45:0/2584600667 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbe8c06c200 con 0x7fbe9c1081c0 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.089+0000 7fbe9b7fe640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fbe84038fc0 0x7fbe8403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.090+0000 7fbe9b7fe640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fbe84038fc0 0x7fbe8403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe900098c0 tx=0x7fbe90006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.094+0000 7fbe997fa640 1 -- 172.21.15.45:0/2584600667 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe8c02cb30 con 0x7fbe9c1081c0 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.272+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fbe9c19f250 con 0x7fbe9c1081c0 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.274+0000 7fbe997fa640 1 -- 172.21.15.45:0/2584600667 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbe8c03b4a0 con 0x7fbe9c1081c0 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.276+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fbe84038fc0 msgr2=0x7fbe8403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.276+0000 7fbea2949640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fbe84038fc0 0x7fbe8403b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbe900098c0 tx=0x7fbe90006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.276+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 msgr2=0x7fbe9c19b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.276+0000 7fbea2949640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c19b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0097f0 tx=0x7fbe8c0327b0 comp rx=0 tx=0).stop 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.277+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 shutdown_connections 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.277+0000 7fbea2949640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fbe84038fc0 0x7fbe8403b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:41.883 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.277+0000 7fbea2949640 1 --2- 172.21.15.45:0/2584600667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe9c1081c0 0x7fbe9c19b730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:41.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.277+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 >> 172.21.15.45:0/2584600667 conn(0x7fbe9c101da0 msgr2=0x7fbe9c1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:41.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.277+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 shutdown_connections 2024-09-10T09:54:41.884 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:41.277+0000 7fbea2949640 1 -- 172.21.15.45:0/2584600667 wait complete. 2024-09-10T09:54:41.884 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-09-10T09:54:43.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-mon[31114]: from='mgr.14118 172.21.15.45:0/3864521296' entity='mgr.x' 2024-09-10T09:54:43.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2351731509' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T09:54:43.650 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ignoring --setuser ceph since I am not root 2024-09-10T09:54:43.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ignoring --setgroup ceph since I am not root 2024-09-10T09:54:43.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:43.555+0000 7ff6ff2a8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:54:43.651 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:43.614+0000 7ff6ff2a8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:54:44.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:43.770+0000 7ff6ff2a8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:54:44.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:43.840+0000 7ff6ff2a8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:54:44.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:43.994+0000 7ff6ff2a8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:54:44.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:44.142+0000 7ff6ff2a8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4fc70640 1 Processor -- start 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4fc70640 1 -- start start 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4fc70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd48107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4fc70640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481080c0 con 0x7fcd481076f0 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4da0c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd48107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4da0c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd48107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45560/0 (socket says 172.21.15.45:45560) 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4da0c640 1 -- 172.21.15.45:0/1217617683 learned_addr learned my addr 172.21.15.45:0/1217617683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.291+0000 7fcd4da0c640 1 -- 172.21.15.45:0/1217617683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd481088f0 con 0x7fcd481076f0 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.292+0000 7fcd4da0c640 1 --2- 172.21.15.45:0/1217617683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd48107af0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009820 tx=0x7fcd4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72862c4c4375b155 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.292+0000 7fcd4ca0a640 1 -- 172.21.15.45:0/1217617683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd44037070 con 0x7fcd481076f0 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.292+0000 7fcd4ca0a640 1 -- 172.21.15.45:0/1217617683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44032920 con 0x7fcd481076f0 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.292+0000 7fcd4ca0a640 1 -- 172.21.15.45:0/1217617683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd44030810 con 0x7fcd481076f0 2024-09-10T09:54:44.459 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.293+0000 7fcd4fc70640 1 -- 172.21.15.45:0/1217617683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 msgr2=0x7fcd48107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.293+0000 7fcd4fc70640 1 --2- 172.21.15.45:0/1217617683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd48107af0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009820 tx=0x7fcd4402a270 comp rx=0 tx=0).stop 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.293+0000 7fcd4fc70640 1 -- 172.21.15.45:0/1217617683 shutdown_connections 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.293+0000 7fcd4fc70640 1 --2- 172.21.15.45:0/1217617683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd48107af0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.293+0000 7fcd4fc70640 1 -- 172.21.15.45:0/1217617683 >> 172.21.15.45:0/1217617683 conn(0x7fcd48102e80 msgr2=0x7fcd481052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.293+0000 7fcd4fc70640 1 -- 172.21.15.45:0/1217617683 shutdown_connections 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.293+0000 7fcd4fc70640 1 -- 172.21.15.45:0/1217617683 wait complete. 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4fc70640 1 Processor -- start 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4fc70640 1 -- start start 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4fc70640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd4807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4fc70640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4810bca0 con 0x7fcd481076f0 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4da0c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd4807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4da0c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd4807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45566/0 (socket says 172.21.15.45:45566) 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4da0c640 1 -- 172.21.15.45:0/2351731509 learned_addr learned my addr 172.21.15.45:0/2351731509 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:44.460 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4da0c640 1 -- 172.21.15.45:0/2351731509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4807e090 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.294+0000 7fcd4da0c640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd4807db50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd44030560 tx=0x7fcd44032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.295+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd44037040 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.295+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd44040070 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.295+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4807c280 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.295+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd4403a740 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.295+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4807c7c0 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.295+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcd4403a8e0 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.296+0000 7fcd3effd640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fcd34038fc0 0x7fcd3403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.296+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd4406c520 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.296+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4810be20 con 0x7fcd481076f0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.296+0000 7fcd4d20b640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fcd34038fc0 0x7fcd3403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:44.461 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.296+0000 7fcd4d20b640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fcd34038fc0 0x7fcd3403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd380098c0 tx=0x7fcd38006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:44.462 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.301+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd44041270 con 0x7fcd481076f0 2024-09-10T09:54:44.462 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:42.538+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcd4807ce70 con 0x7fcd481076f0 2024-09-10T09:54:44.462 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.397+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcd44045980 con 0x7fcd481076f0 2024-09-10T09:54:44.462 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.399+0000 7fcd3effd640 1 -- 172.21.15.45:0/2351731509 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fcd44045460 con 0x7fcd481076f0 2024-09-10T09:54:44.462 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.400+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fcd34038fc0 msgr2=0x7fcd3403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.462 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.400+0000 7fcd4fc70640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fcd34038fc0 0x7fcd3403b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcd380098c0 tx=0x7fcd38006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.462 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 msgr2=0x7fcd4807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:44.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd4807db50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd44030560 tx=0x7fcd44032670 comp rx=0 tx=0).stop 2024-09-10T09:54:44.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 shutdown_connections 2024-09-10T09:54:44.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fcd34038fc0 0x7fcd3403b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 --2- 172.21.15.45:0/2351731509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd481076f0 0x7fcd4807db50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:44.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 >> 172.21.15.45:0/2351731509 conn(0x7fcd48102e80 msgr2=0x7fcd48103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:44.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 shutdown_connections 2024-09-10T09:54:44.463 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:43.401+0000 7fcd4fc70640 1 -- 172.21.15.45:0/2351731509 wait complete. 2024-09-10T09:54:44.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:44 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2351731509' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T09:54:44.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:44 smithi045 ceph-mon[31114]: mgrmap e9: x(active, since 19s) 2024-09-10T09:54:44.650 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:44.478+0000 7ff6ff2a8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:54:44.650 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:44.542+0000 7ff6ff2a8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:54:45.126 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:44.741+0000 7ff6ff2a8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:54:45.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:45 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1820966910' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.918+0000 7f48e29a8640 1 Processor -- start 2024-09-10T09:54:45.800 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.918+0000 7f48e29a8640 1 -- start start 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.919+0000 7f48e29a8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc10a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.919+0000 7f48e29a8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc07be90 con 0x7f48dc108160 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.919+0000 7f48dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc10a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.919+0000 7f48dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc10a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59870/0 (socket says 172.21.15.45:59870) 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.919+0000 7f48dbfff640 1 -- 172.21.15.45:0/45036694 learned_addr learned my addr 172.21.15.45:0/45036694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.919+0000 7f48dbfff640 1 -- 172.21.15.45:0/45036694 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc10aa90 con 0x7f48dc108160 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.920+0000 7f48dbfff640 1 --2- 172.21.15.45:0/45036694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc10a550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009820 tx=0x7f48cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74738c55ca38528e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.920+0000 7f48daffd640 1 -- 172.21.15.45:0/45036694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc037070 con 0x7f48dc108160 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.920+0000 7f48daffd640 1 -- 172.21.15.45:0/45036694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc032920 con 0x7f48dc108160 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.920+0000 7f48daffd640 1 -- 172.21.15.45:0/45036694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc030810 con 0x7f48dc108160 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.921+0000 7f48e29a8640 1 -- 172.21.15.45:0/45036694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 msgr2=0x7f48dc10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.921+0000 7f48e29a8640 1 --2- 172.21.15.45:0/45036694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc10a550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f48cc009820 tx=0x7f48cc02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.921+0000 7f48e29a8640 1 -- 172.21.15.45:0/45036694 shutdown_connections 2024-09-10T09:54:45.801 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.921+0000 7f48e29a8640 1 --2- 172.21.15.45:0/45036694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc10a550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.921+0000 7f48e29a8640 1 -- 172.21.15.45:0/45036694 >> 172.21.15.45:0/45036694 conn(0x7f48dc101da0 msgr2=0x7f48dc1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.921+0000 7f48e29a8640 1 -- 172.21.15.45:0/45036694 shutdown_connections 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.921+0000 7f48e29a8640 1 -- 172.21.15.45:0/45036694 wait complete. 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48e29a8640 1 Processor -- start 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48e29a8640 1 -- start start 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48e29a8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc1079e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48e29a8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f48dc10bb70 con 0x7f48dc108160 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc1079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc1079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59874/0 (socket says 172.21.15.45:59874) 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48dbfff640 1 -- 172.21.15.45:0/1820966910 learned_addr learned my addr 172.21.15.45:0/1820966910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.922+0000 7f48dbfff640 1 -- 172.21.15.45:0/1820966910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48dc107f20 con 0x7f48dc108160 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.923+0000 7f48dbfff640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc1079e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0097f0 tx=0x7f48cc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.923+0000 7f48d97fa640 1 -- 172.21.15.45:0/1820966910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc037040 con 0x7f48dc108160 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.923+0000 7f48d97fa640 1 -- 172.21.15.45:0/1820966910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc030c40 con 0x7f48dc108160 2024-09-10T09:54:45.802 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.923+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48dc106110 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.923+0000 7f48d97fa640 1 -- 172.21.15.45:0/1820966910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48cc03a5d0 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.923+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48dc106650 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.924+0000 7f48d97fa640 1 -- 172.21.15.45:0/1820966910 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f48cc03a7c0 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.924+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48dc1085c0 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.924+0000 7f48d97fa640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 0x7f48c403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.924+0000 7f48d97fa640 1 -- 172.21.15.45:0/1820966910 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f48cc06c470 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.924+0000 7f48db7fe640 1 -- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 msgr2=0x7f48c403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1958743370 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.924+0000 7f48db7fe640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 0x7f48c403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:44.929+0000 7f48d97fa640 1 -- 172.21.15.45:0/1820966910 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48cc030df0 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.125+0000 7f48db7fe640 1 -- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 msgr2=0x7f48c403b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1958743370 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.125+0000 7f48db7fe640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 0x7f48c403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.153+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f48dc106df0 con 0x7f48dc108160 2024-09-10T09:54:45.803 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.154+0000 7f48d97fa640 1 -- 172.21.15.45:0/1820966910 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f48cc02fd60 con 0x7f48dc108160 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 msgr2=0x7f48c403b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 0x7f48c403b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 msgr2=0x7f48dc1079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc1079e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0097f0 tx=0x7f48cc0327b0 comp rx=0 tx=0).stop 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 shutdown_connections 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7f48c4039020 0x7f48c403b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 --2- 172.21.15.45:0/1820966910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f48dc108160 0x7f48dc1079e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 >> 172.21.15.45:0/1820966910 conn(0x7f48dc101da0 msgr2=0x7f48dc1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 shutdown_connections 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:45.156+0000 7f48e29a8640 1 -- 172.21.15.45:0/1820966910 wait complete. 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-09-10T09:54:45.804 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-09-10T09:54:45.926 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:45.704+0000 7ff6ff2a8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:54:45.926 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:45.777+0000 7ff6ff2a8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:54:46.831 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:46.559+0000 7ff6ff2a8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:54:46.831 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:46.680+0000 7ff6ff2a8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:54:47.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:46.830+0000 7ff6ff2a8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:54:47.124 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:46.894+0000 7ff6ff2a8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:54:47.125 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.052+0000 7ff6ff2a8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:54:47.125 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.123+0000 7ff6ff2a8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:54:47.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.186+0000 7ff6ff2a8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:54:47.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.352+0000 7ff6ff2a8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:54:47.702 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.420+0000 7ff6ff2a8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:54:47.702 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.490+0000 7ff6ff2a8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:54:47.702 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.577+0000 7ff6ff2a8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:54:47.703 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.639+0000 7ff6ff2a8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:54:47.703 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:47.701+0000 7ff6ff2a8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:54:48.799 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:48.531+0000 7ff6ff2a8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:54:48.799 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: Active manager daemon x restarted 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: Activating manager daemon x 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: mgrmap e10: x(active, starting, since 0.00309971s) 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: Manager daemon x is now available 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:54:48.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.244+0000 7fc8ab91b640 1 Processor -- start 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.244+0000 7fc8ab91b640 1 -- start start 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.244+0000 7fc8ab91b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a4106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.244+0000 7fc8ab91b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4107000 con 0x7fc8a4106630 2024-09-10T09:54:50.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.244+0000 7fc8a96b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a4106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.244+0000 7fc8a96b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a4106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59886/0 (socket says 172.21.15.45:59886) 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.244+0000 7fc8a96b7640 1 -- 172.21.15.45:0/323298555 learned_addr learned my addr 172.21.15.45:0/323298555 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.245+0000 7fc8a96b7640 1 -- 172.21.15.45:0/323298555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a4107830 con 0x7fc8a4106630 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.246+0000 7fc8a96b7640 1 --2- 172.21.15.45:0/323298555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a4106a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009820 tx=0x7fc8a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b046e43ebb4cb792 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.246+0000 7fc897fff640 1 -- 172.21.15.45:0/323298555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a0037070 con 0x7fc8a4106630 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.246+0000 7fc897fff640 1 -- 172.21.15.45:0/323298555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8a0032920 con 0x7fc8a4106630 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.246+0000 7fc897fff640 1 -- 172.21.15.45:0/323298555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a0030810 con 0x7fc8a4106630 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 -- 172.21.15.45:0/323298555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 msgr2=0x7fc8a4106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 --2- 172.21.15.45:0/323298555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a4106a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009820 tx=0x7fc8a002a270 comp rx=0 tx=0).stop 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 -- 172.21.15.45:0/323298555 shutdown_connections 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 --2- 172.21.15.45:0/323298555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a4106a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 -- 172.21.15.45:0/323298555 >> 172.21.15.45:0/323298555 conn(0x7fc8a4101e20 msgr2=0x7fc8a4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:50.445 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 -- 172.21.15.45:0/323298555 shutdown_connections 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 -- 172.21.15.45:0/323298555 wait complete. 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.247+0000 7fc8ab91b640 1 Processor -- start 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8ab91b640 1 -- start start 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8ab91b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8ab91b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a410abe0 con 0x7fc8a4106630 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8a96b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8a96b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59898/0 (socket says 172.21.15.45:59898) 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8a96b7640 1 -- 172.21.15.45:0/3887144747 learned_addr learned my addr 172.21.15.45:0/3887144747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8a96b7640 1 -- 172.21.15.45:0/3887144747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419df40 con 0x7fc8a4106630 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.248+0000 7fc8a96b7640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a419da00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00097f0 tx=0x7fc8a00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.249+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a0037040 con 0x7fc8a4106630 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.249+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8a0030c40 con 0x7fc8a4106630 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.249+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a419e1d0 con 0x7fc8a4106630 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.249+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a003a5b0 con 0x7fc8a4106630 2024-09-10T09:54:50.446 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.249+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a419e5f0 con 0x7fc8a4106630 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.250+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc8a003a7c0 con 0x7fc8a4106630 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.250+0000 7fc8967fc640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 0x7fc89003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.250+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 --> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc89003bbf0 con 0x7fc890039020 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.250+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc8a006c4f0 con 0x7fc8a4106630 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.250+0000 7fc8a8eb6640 1 -- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 msgr2=0x7fc89003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1958743370 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.250+0000 7fc8a8eb6640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 0x7fc89003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.451+0000 7fc8a8eb6640 1 -- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 msgr2=0x7fc89003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1958743370 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.451+0000 7fc8a8eb6640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 0x7fc89003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.852+0000 7fc8a8eb6640 1 -- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 msgr2=0x7fc89003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1958743370 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:46.852+0000 7fc8a8eb6640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 0x7fc89003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:47.653+0000 7fc8a8eb6640 1 -- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 msgr2=0x7fc89003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1958743370 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:47.653+0000 7fc8a8eb6640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 0x7fc89003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:48.534+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fc8a006afa0 con 0x7fc8a4106630 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:48.535+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 msgr2=0x7fc89003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:54:50.447 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:48.535+0000 7fc8967fc640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 0x7fc89003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.535+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fc8a003ba10 con 0x7fc8a4106630 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.535+0000 7fc8967fc640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc89003cae0 0x7fc89003eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.535+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc89003bbf0 con 0x7fc89003cae0 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.535+0000 7fc8a8eb6640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc89003cae0 0x7fc89003eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.536+0000 7fc8a8eb6640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc89003cae0 0x7fc89003eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc898003980 tx=0x7fc8980092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.537+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fc89003bbf0 con 0x7fc89003cae0 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.545+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc8a410ad60 con 0x7fc89003cae0 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.545+0000 7fc8967fc640 1 -- 172.21.15.45:0/3887144747 <== mgr.14150 v2:172.21.15.45:6800/952575454 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fc8a410ad60 con 0x7fc89003cae0 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.545+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc89003cae0 msgr2=0x7fc89003eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.545+0000 7fc8ab91b640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc89003cae0 0x7fc89003eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc898003980 tx=0x7fc8980092b0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.545+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 msgr2=0x7fc8a419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a419da00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00097f0 tx=0x7fc8a00327b0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 shutdown_connections 2024-09-10T09:54:50.448 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc89003cae0 0x7fc89003eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:6800/1958743370,v1:172.21.15.45:6801/1958743370] conn(0x7fc890039020 0x7fc89003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 --2- 172.21.15.45:0/3887144747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a4106630 0x7fc8a419da00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 >> 172.21.15.45:0/3887144747 conn(0x7fc8a4101e20 msgr2=0x7fc8a4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 shutdown_connections 2024-09-10T09:54:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:49.546+0000 7fc8ab91b640 1 -- 172.21.15.45:0/3887144747 wait complete. 2024-09-10T09:54:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-09-10T09:54:50.449 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-09-10T09:54:50.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-mon[31114]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T09:54:50.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-mon[31114]: mgrmap e11: x(active, since 1.00574s) 2024-09-10T09:54:50.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-mon[31114]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T09:54:50.792 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:54:51.049 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:50.789+0000 7ff66a57c640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:09:54:50] ENGINE Error in HTTPServer.serve 2024-09-10T09:54:51.049 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: Traceback (most recent call last): 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._connections.run(self.expiration_interval) 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._run(expiration_interval) 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: s = self.context.wrap_socket( 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:54:51.050 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: return self.sslsocket_class._create( 2024-09-10T09:54:51.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:54:51.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self.do_handshake() 2024-09-10T09:54:51.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:54:51.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._sslobj.do_handshake() 2024-09-10T09:54:51.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:54:51.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 09:54:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T09:54:51.708 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T09:54:51.708 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.852+0000 7fcb43af2640 1 Processor -- start 2024-09-10T09:54:51.708 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.852+0000 7fcb43af2640 1 -- start start 2024-09-10T09:54:51.708 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.852+0000 7fcb43af2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c10a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:51.708 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.852+0000 7fcb43af2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c07be30 con 0x7fcb3c108200 2024-09-10T09:54:51.708 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.852+0000 7fcb4188e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c10a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.853+0000 7fcb4188e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c10a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59960/0 (socket says 172.21.15.45:59960) 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.853+0000 7fcb4188e640 1 -- 172.21.15.45:0/640084488 learned_addr learned my addr 172.21.15.45:0/640084488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.853+0000 7fcb4188e640 1 -- 172.21.15.45:0/640084488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c10ab30 con 0x7fcb3c108200 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb4188e640 1 --2- 172.21.15.45:0/640084488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c10a5f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009820 tx=0x7fcb3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8227ef981229ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb4088c640 1 -- 172.21.15.45:0/640084488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb38037070 con 0x7fcb3c108200 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb4088c640 1 -- 172.21.15.45:0/640084488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb38032920 con 0x7fcb3c108200 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb4088c640 1 -- 172.21.15.45:0/640084488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb38030810 con 0x7fcb3c108200 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb43af2640 1 -- 172.21.15.45:0/640084488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 msgr2=0x7fcb3c10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb43af2640 1 --2- 172.21.15.45:0/640084488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c10a5f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009820 tx=0x7fcb3802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb43af2640 1 -- 172.21.15.45:0/640084488 shutdown_connections 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb43af2640 1 --2- 172.21.15.45:0/640084488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c10a5f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb43af2640 1 -- 172.21.15.45:0/640084488 >> 172.21.15.45:0/640084488 conn(0x7fcb3c101e20 msgr2=0x7fcb3c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:51.709 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb43af2640 1 -- 172.21.15.45:0/640084488 shutdown_connections 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.854+0000 7fcb43af2640 1 -- 172.21.15.45:0/640084488 wait complete. 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.855+0000 7fcb43af2640 1 Processor -- start 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.855+0000 7fcb43af2640 1 -- start start 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.855+0000 7fcb43af2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.855+0000 7fcb43af2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c10bbb0 con 0x7fcb3c108200 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.855+0000 7fcb4188e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.855+0000 7fcb4188e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59968/0 (socket says 172.21.15.45:59968) 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.855+0000 7fcb4188e640 1 -- 172.21.15.45:0/592860620 learned_addr learned my addr 172.21.15.45:0/592860620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.856+0000 7fcb4188e640 1 -- 172.21.15.45:0/592860620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c107fc0 con 0x7fcb3c108200 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.856+0000 7fcb4188e640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c107a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb38030420 tx=0x7fcb38032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.856+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb38037040 con 0x7fcb3c108200 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.856+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb38040070 con 0x7fcb3c108200 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.856+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c1061b0 con 0x7fcb3c108200 2024-09-10T09:54:51.710 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.856+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3803a6e0 con 0x7fcb3c108200 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.856+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1066f0 con 0x7fcb3c108200 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.857+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fcb3803bb20 con 0x7fcb3c108200 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.857+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c108660 con 0x7fcb3c108200 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.857+0000 7fcb32ffd640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcb28038eb0 0x7fcb2803b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.857+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb3806c570 con 0x7fcb3c108200 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.857+0000 7fcb4108d640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcb28038eb0 0x7fcb2803b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.858+0000 7fcb4108d640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcb28038eb0 0x7fcb2803b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098c0 tx=0x7fcb2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.862+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb3803b7a0 con 0x7fcb3c108200 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:50.897+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fcb38031ad0 con 0x7fcb3c108200 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.047+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fcb3c107040 con 0x7fcb28038eb0 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.089+0000 7fcb32ffd640 1 -- 172.21.15.45:0/592860620 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fcb3c107040 con 0x7fcb28038eb0 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcb28038eb0 msgr2=0x7fcb2803b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:51.711 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcb28038eb0 0x7fcb2803b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0098c0 tx=0x7fcb2c006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 msgr2=0x7fcb3c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c107a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb38030420 tx=0x7fcb38032670 comp rx=0 tx=0).stop 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 shutdown_connections 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcb28038eb0 0x7fcb2803b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 --2- 172.21.15.45:0/592860620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3c108200 0x7fcb3c107a80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 >> 172.21.15.45:0/592860620 conn(0x7fcb3c101e20 msgr2=0x7fcb3c102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 shutdown_connections 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:51.093+0000 7fcb43af2640 1 -- 172.21.15.45:0/592860620 wait complete. 2024-09-10T09:54:51.712 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-09-10T09:54:51.952 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:50] ENGINE Bus STARTING 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:50] ENGINE Serving on https://172.21.15.45:7150 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:50] ENGINE Error in HTTPServer.serve 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: Traceback (most recent call last): 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._connections.run(self.expiration_interval) 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._run(expiration_interval) 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:54:51.953 INFO:journalctl@ceph.mon.a.smithi045.stdout: s = self.context.wrap_socket( 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout: return self.sslsocket_class._create( 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout: self.do_handshake() 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._sslobj.do_handshake() 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:50] ENGINE Serving on http://172.21.15.45:8765 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: [10/Sep/2024:09:54:50] ENGINE Bus STARTED 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: mgrmap e12: x(active, since 2s) 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:54:51.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:54:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:52 smithi045 ceph-mon[31114]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:54:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:52 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:54:53.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$3VH5uF0cd0kGep/BP8u8me6y3cF37zUR/dxgBJsCjiUsaNyVLk6dK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725962092, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T09:54:53.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.140+0000 7fc977ea1640 1 Processor -- start 2024-09-10T09:54:53.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.140+0000 7fc977ea1640 1 -- start start 2024-09-10T09:54:53.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.140+0000 7fc977ea1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:53.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.140+0000 7fc977ea1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc97007c8b0 con 0x7fc97007de80 2024-09-10T09:54:53.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.141+0000 7fc975c3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:53.694 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.141+0000 7fc975c3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59972/0 (socket says 172.21.15.45:59972) 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.141+0000 7fc975c3d640 1 -- 172.21.15.45:0/2918470378 learned_addr learned my addr 172.21.15.45:0/2918470378 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.141+0000 7fc975c3d640 1 -- 172.21.15.45:0/2918470378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97007ca30 con 0x7fc97007de80 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.142+0000 7fc975c3d640 1 --2- 172.21.15.45:0/2918470378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97007c2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009820 tx=0x7fc96c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55ddb69346bd459 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.142+0000 7fc974c3b640 1 -- 172.21.15.45:0/2918470378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c037070 con 0x7fc97007de80 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.142+0000 7fc974c3b640 1 -- 172.21.15.45:0/2918470378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c032920 con 0x7fc97007de80 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.142+0000 7fc974c3b640 1 -- 172.21.15.45:0/2918470378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c030810 con 0x7fc97007de80 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.142+0000 7fc977ea1640 1 -- 172.21.15.45:0/2918470378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 msgr2=0x7fc97007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.142+0000 7fc977ea1640 1 --2- 172.21.15.45:0/2918470378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97007c2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009820 tx=0x7fc96c02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 -- 172.21.15.45:0/2918470378 shutdown_connections 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 --2- 172.21.15.45:0/2918470378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97007c2e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 -- 172.21.15.45:0/2918470378 >> 172.21.15.45:0/2918470378 conn(0x7fc970102e80 msgr2=0x7fc9701052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 -- 172.21.15.45:0/2918470378 shutdown_connections 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 -- 172.21.15.45:0/2918470378 wait complete. 2024-09-10T09:54:53.695 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 Processor -- start 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 -- start start 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.143+0000 7fc977ea1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc97010ddc0 con 0x7fc97007de80 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc975c3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc975c3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59986/0 (socket says 172.21.15.45:59986) 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc975c3d640 1 -- 172.21.15.45:0/4238471026 learned_addr learned my addr 172.21.15.45:0/4238471026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc975c3d640 1 -- 172.21.15.45:0/4238471026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97019de50 con 0x7fc97007de80 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc975c3d640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97019d910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0097f0 tx=0x7fc96c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc966ffd640 1 -- 172.21.15.45:0/4238471026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c037040 con 0x7fc97007de80 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc966ffd640 1 -- 172.21.15.45:0/4238471026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc96c030cc0 con 0x7fc97007de80 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97019e0e0 con 0x7fc97007de80 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc966ffd640 1 -- 172.21.15.45:0/4238471026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc96c03a6b0 con 0x7fc97007de80 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.144+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9701a0dd0 con 0x7fc97007de80 2024-09-10T09:54:53.696 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.145+0000 7fc966ffd640 1 -- 172.21.15.45:0/4238471026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fc96c03a850 con 0x7fc97007de80 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.146+0000 7fc966ffd640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc95c039020 0x7fc95c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.146+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97007c360 con 0x7fc97007de80 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.146+0000 7fc966ffd640 1 -- 172.21.15.45:0/4238471026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc96c06c760 con 0x7fc97007de80 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.146+0000 7fc97543c640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc95c039020 0x7fc95c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.146+0000 7fc97543c640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc95c039020 0x7fc95c03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9600098c0 tx=0x7fc960006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.150+0000 7fc966ffd640 1 -- 172.21.15.45:0/4238471026 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc96c044920 con 0x7fc97007de80 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.348+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc97019e5c0 con 0x7fc95c039020 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.597+0000 7fc966ffd640 1 -- 172.21.15.45:0/4238471026 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc97019e5c0 con 0x7fc95c039020 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc95c039020 msgr2=0x7fc95c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc95c039020 0x7fc95c03b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9600098c0 tx=0x7fc960006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 msgr2=0x7fc97019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97019d910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0097f0 tx=0x7fc96c0327b0 comp rx=0 tx=0).stop 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 shutdown_connections 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc95c039020 0x7fc95c03b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:53.697 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 --2- 172.21.15.45:0/4238471026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc97007de80 0x7fc97019d910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:53.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 >> 172.21.15.45:0/4238471026 conn(0x7fc970102e80 msgr2=0x7fc970103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:53.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 shutdown_connections 2024-09-10T09:54:53.698 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:52.600+0000 7fc977ea1640 1 -- 172.21.15.45:0/4238471026 wait complete. 2024-09-10T09:54:53.698 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-09-10T09:54:54.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:54 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2515913889' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T09:54:54.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T09:54:54.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa03d497640 1 Processor -- start 2024-09-10T09:54:54.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa03d497640 1 -- start start 2024-09-10T09:54:54.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa03d497640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03810a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:54.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa03d497640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa03807be10 con 0x7fa038108200 2024-09-10T09:54:54.926 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa036ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03810a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa036ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03810a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40658/0 (socket says 172.21.15.45:40658) 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa036ffd640 1 -- 172.21.15.45:0/3620639060 learned_addr learned my addr 172.21.15.45:0/3620639060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.152+0000 7fa036ffd640 1 -- 172.21.15.45:0/3620639060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03810ab30 con 0x7fa038108200 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.153+0000 7fa036ffd640 1 --2- 172.21.15.45:0/3620639060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03810a5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa028009820 tx=0x7fa02802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119f16fd00c06fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.153+0000 7fa035ffb640 1 -- 172.21.15.45:0/3620639060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa028037070 con 0x7fa038108200 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.153+0000 7fa035ffb640 1 -- 172.21.15.45:0/3620639060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa028032920 con 0x7fa038108200 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.153+0000 7fa035ffb640 1 -- 172.21.15.45:0/3620639060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa028030810 con 0x7fa038108200 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.154+0000 7fa03d497640 1 -- 172.21.15.45:0/3620639060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 msgr2=0x7fa03810a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.154+0000 7fa03d497640 1 --2- 172.21.15.45:0/3620639060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03810a5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa028009820 tx=0x7fa02802a270 comp rx=0 tx=0).stop 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.154+0000 7fa03d497640 1 -- 172.21.15.45:0/3620639060 shutdown_connections 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.154+0000 7fa03d497640 1 --2- 172.21.15.45:0/3620639060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03810a5f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.927 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.154+0000 7fa03d497640 1 -- 172.21.15.45:0/3620639060 >> 172.21.15.45:0/3620639060 conn(0x7fa038101da0 msgr2=0x7fa0381041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.154+0000 7fa03d497640 1 -- 172.21.15.45:0/3620639060 shutdown_connections 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.154+0000 7fa03d497640 1 -- 172.21.15.45:0/3620639060 wait complete. 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa03d497640 1 Processor -- start 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa03d497640 1 -- start start 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa03d497640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa03d497640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa03810bb50 con 0x7fa038108200 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa036ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa036ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03819b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40670/0 (socket says 172.21.15.45:40670) 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa036ffd640 1 -- 172.21.15.45:0/2515913889 learned_addr learned my addr 172.21.15.45:0/2515913889 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.155+0000 7fa036ffd640 1 -- 172.21.15.45:0/2515913889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03819bc70 con 0x7fa038108200 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.156+0000 7fa036ffd640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03819b730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa028030380 tx=0x7fa028032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.156+0000 7fa01ffff640 1 -- 172.21.15.45:0/2515913889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa028037040 con 0x7fa038108200 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.156+0000 7fa01ffff640 1 -- 172.21.15.45:0/2515913889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa028040070 con 0x7fa038108200 2024-09-10T09:54:54.928 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.156+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa03819bf00 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.156+0000 7fa01ffff640 1 -- 172.21.15.45:0/2515913889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa02803a7b0 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.156+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03819ebf0 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.157+0000 7fa01ffff640 1 -- 172.21.15.45:0/2515913889 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fa02803a9e0 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.157+0000 7fa01ffff640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa020039020 0x7fa02003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.157+0000 7fa01ffff640 1 -- 172.21.15.45:0/2515913889 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa02806c710 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.157+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa038108660 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.157+0000 7fa0367fc640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa020039020 0x7fa02003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.157+0000 7fa0367fc640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa020039020 0x7fa02003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0098c0 tx=0x7fa02c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.162+0000 7fa01ffff640 1 -- 172.21.15.45:0/2515913889 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa028044c70 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.336+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa03819edc0 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.336+0000 7fa01ffff640 1 -- 172.21.15.45:0/2515913889 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa028041270 con 0x7fa038108200 2024-09-10T09:54:54.929 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa020039020 msgr2=0x7fa02003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa020039020 0x7fa02003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0098c0 tx=0x7fa02c006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 msgr2=0x7fa03819b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03819b730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa028030380 tx=0x7fa028032670 comp rx=0 tx=0).stop 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 shutdown_connections 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa020039020 0x7fa02003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 --2- 172.21.15.45:0/2515913889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa038108200 0x7fa03819b730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 >> 172.21.15.45:0/2515913889 conn(0x7fa038101da0 msgr2=0x7fa038102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 shutdown_connections 2024-09-10T09:54:54.930 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:54.339+0000 7fa03d497640 1 -- 172.21.15.45:0/2515913889 wait complete. 2024-09-10T09:54:54.934 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:54:54.935 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-09-10T09:54:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:54:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-09-10T09:54:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-09-10T09:54:54.943 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T09:54:54.945 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-09-10T09:54:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-09-10T09:54:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-09-10T09:54:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: Password: h1opld6ae3 2024-09-10T09:54:54.945 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:54.945 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config directory 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126f243640 1 Processor -- start 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126f243640 1 -- start start 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126f243640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f1268107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126f243640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12681080c0 con 0x7f12681076f0 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126cfdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f1268107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126cfdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f1268107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40678/0 (socket says 172.21.15.45:40678) 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126cfdf640 1 -- 172.21.15.45:0/363633946 learned_addr learned my addr 172.21.15.45:0/363633946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:56.181 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.343+0000 7f126cfdf640 1 -- 172.21.15.45:0/363633946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681088f0 con 0x7f12681076f0 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.344+0000 7f126cfdf640 1 --2- 172.21.15.45:0/363633946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f1268107af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f125c009820 tx=0x7f125c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92295add69a6b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.344+0000 7f12677fe640 1 -- 172.21.15.45:0/363633946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c037070 con 0x7f12681076f0 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.344+0000 7f12677fe640 1 -- 172.21.15.45:0/363633946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c032920 con 0x7f12681076f0 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.344+0000 7f12677fe640 1 -- 172.21.15.45:0/363633946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c030810 con 0x7f12681076f0 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 -- 172.21.15.45:0/363633946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 msgr2=0x7f1268107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 --2- 172.21.15.45:0/363633946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f1268107af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f125c009820 tx=0x7f125c02a270 comp rx=0 tx=0).stop 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 -- 172.21.15.45:0/363633946 shutdown_connections 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 --2- 172.21.15.45:0/363633946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f1268107af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 -- 172.21.15.45:0/363633946 >> 172.21.15.45:0/363633946 conn(0x7f1268102e80 msgr2=0x7f12681052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 -- 172.21.15.45:0/363633946 shutdown_connections 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 -- 172.21.15.45:0/363633946 wait complete. 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.345+0000 7f126f243640 1 Processor -- start 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126f243640 1 -- start start 2024-09-10T09:54:56.182 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126f243640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f126807db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126f243640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f126810bca0 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126cfdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f126807db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126cfdf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f126807db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40692/0 (socket says 172.21.15.45:40692) 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126cfdf640 1 -- 172.21.15.45:0/2947052152 learned_addr learned my addr 172.21.15.45:0/2947052152 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126cfdf640 1 -- 172.21.15.45:0/2947052152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126807e090 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126cfdf640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f126807db50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f125c0097f0 tx=0x7f125c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c037040 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125c030cc0 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f125c03a640 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.346+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126807c280 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.347+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126807c7c0 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.347+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f125c03b3f0 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.348+0000 7f1265ffb640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1258039020 0x7f125803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.348+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126810be20 con 0x7f12681076f0 2024-09-10T09:54:56.183 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.348+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f125c06c6f0 con 0x7f12681076f0 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.348+0000 7f1267fff640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1258039020 0x7f125803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.348+0000 7f1267fff640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1258039020 0x7f125803b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f12540098c0 tx=0x7f1254006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.352+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f125c042e10 con 0x7f12681076f0 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.390+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f125c049ca0 con 0x7f12681076f0 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.619+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1268107b70 con 0x7f12681076f0 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.622+0000 7f1265ffb640 1 -- 172.21.15.45:0/2947052152 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f125c03f1c0 con 0x7f12681076f0 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1258039020 msgr2=0x7f125803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1258039020 0x7f125803b4e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f12540098c0 tx=0x7f1254006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 msgr2=0x7f126807db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f126807db50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f125c0097f0 tx=0x7f125c0327b0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 shutdown_connections 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1258039020 0x7f125803b4e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 --2- 172.21.15.45:0/2947052152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12681076f0 0x7f126807db50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:56.184 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 >> 172.21.15.45:0/2947052152 conn(0x7f1268102e80 msgr2=0x7f1268103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 shutdown_connections 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-09-10T09:54:55.625+0000 7f126f243640 1 -- 172.21.15.45:0/2947052152 wait complete. 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-09-10T09:54:56.185 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.186 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T09:54:56.186 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:54:56.186 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-09-10T09:54:56.220 INFO:tasks.cephadm:Fetching config... 2024-09-10T09:54:56.221 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:54:56.221 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T09:54:56.237 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T09:54:56.237 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:54:56.238 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T09:54:56.293 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T09:54:56.294 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:54:56.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T09:54:56.359 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T09:54:56.359 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:54:56.359 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T09:54:56.414 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T09:54:56.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8IApLOVeOHtvaZle40hZ/i0UuxQbbFTC2FDoS2YjSGl5ARK5OjJMF8fQapT/Y9ffgfu/Aymmo3b77dv74EJ2EG4jIxeoAWmuJHs8ILxOU89jE58WBb1BpYx83q4IUwdiVotUAIi6JSOTG67NWOkerc6NZ2kMe9UbLC/KILR07SCYRV5gInVzlcNlC2Pb7v5g46S5OCzNLivVD1pEK6XJSiDk8pLFd7z9jL2sXn/OFiQQKWZ3b0wIR4+qbJrezLQJYBE0RGAT28yuWxVZgkMIKDtl/2UCvk06SZzwj+JqGni491gKlbU1qyJvMattCC9/O11tcd/NzBb5Ugr+Kg3PoH+drpT0C2Y7DJwovaxTtIMBkXCd1wq4n1J4EVN/RVpkelhTf6N2sf9Rwcb1I7740xh4x//srh0yZpz4/sJP0XzWmLAks+e1EHWhTDsWG2Un+Dwtaa71KbxKrQSHKYD8TSJmxXrSCJ6KTRP0pGfLa7xDRbQM97LOEeF89Dshle1c= ceph-729b10fe-6f5a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:54:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:56 smithi045 ceph-mon[31114]: mgrmap e13: x(active, since 6s) 2024-09-10T09:54:56.477 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:56 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2947052152' entity='client.admin' 2024-09-10T09:54:56.492 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8IApLOVeOHtvaZle40hZ/i0UuxQbbFTC2FDoS2YjSGl5ARK5OjJMF8fQapT/Y9ffgfu/Aymmo3b77dv74EJ2EG4jIxeoAWmuJHs8ILxOU89jE58WBb1BpYx83q4IUwdiVotUAIi6JSOTG67NWOkerc6NZ2kMe9UbLC/KILR07SCYRV5gInVzlcNlC2Pb7v5g46S5OCzNLivVD1pEK6XJSiDk8pLFd7z9jL2sXn/OFiQQKWZ3b0wIR4+qbJrezLQJYBE0RGAT28yuWxVZgkMIKDtl/2UCvk06SZzwj+JqGni491gKlbU1qyJvMattCC9/O11tcd/NzBb5Ugr+Kg3PoH+drpT0C2Y7DJwovaxTtIMBkXCd1wq4n1J4EVN/RVpkelhTf6N2sf9Rwcb1I7740xh4x//srh0yZpz4/sJP0XzWmLAks+e1EHWhTDsWG2Un+Dwtaa71KbxKrQSHKYD8TSJmxXrSCJ6KTRP0pGfLa7xDRbQM97LOEeF89Dshle1c= ceph-729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:56.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8IApLOVeOHtvaZle40hZ/i0UuxQbbFTC2FDoS2YjSGl5ARK5OjJMF8fQapT/Y9ffgfu/Aymmo3b77dv74EJ2EG4jIxeoAWmuJHs8ILxOU89jE58WBb1BpYx83q4IUwdiVotUAIi6JSOTG67NWOkerc6NZ2kMe9UbLC/KILR07SCYRV5gInVzlcNlC2Pb7v5g46S5OCzNLivVD1pEK6XJSiDk8pLFd7z9jL2sXn/OFiQQKWZ3b0wIR4+qbJrezLQJYBE0RGAT28yuWxVZgkMIKDtl/2UCvk06SZzwj+JqGni491gKlbU1qyJvMattCC9/O11tcd/NzBb5Ugr+Kg3PoH+drpT0C2Y7DJwovaxTtIMBkXCd1wq4n1J4EVN/RVpkelhTf6N2sf9Rwcb1I7740xh4x//srh0yZpz4/sJP0XzWmLAks+e1EHWhTDsWG2Un+Dwtaa71KbxKrQSHKYD8TSJmxXrSCJ6KTRP0pGfLa7xDRbQM97LOEeF89Dshle1c= ceph-729b10fe-6f5a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:54:56.548 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8IApLOVeOHtvaZle40hZ/i0UuxQbbFTC2FDoS2YjSGl5ARK5OjJMF8fQapT/Y9ffgfu/Aymmo3b77dv74EJ2EG4jIxeoAWmuJHs8ILxOU89jE58WBb1BpYx83q4IUwdiVotUAIi6JSOTG67NWOkerc6NZ2kMe9UbLC/KILR07SCYRV5gInVzlcNlC2Pb7v5g46S5OCzNLivVD1pEK6XJSiDk8pLFd7z9jL2sXn/OFiQQKWZ3b0wIR4+qbJrezLQJYBE0RGAT28yuWxVZgkMIKDtl/2UCvk06SZzwj+JqGni491gKlbU1qyJvMattCC9/O11tcd/NzBb5Ugr+Kg3PoH+drpT0C2Y7DJwovaxTtIMBkXCd1wq4n1J4EVN/RVpkelhTf6N2sf9Rwcb1I7740xh4x//srh0yZpz4/sJP0XzWmLAks+e1EHWhTDsWG2Un+Dwtaa71KbxKrQSHKYD8TSJmxXrSCJ6KTRP0pGfLa7xDRbQM97LOEeF89Dshle1c= ceph-729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:56.561 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8IApLOVeOHtvaZle40hZ/i0UuxQbbFTC2FDoS2YjSGl5ARK5OjJMF8fQapT/Y9ffgfu/Aymmo3b77dv74EJ2EG4jIxeoAWmuJHs8ILxOU89jE58WBb1BpYx83q4IUwdiVotUAIi6JSOTG67NWOkerc6NZ2kMe9UbLC/KILR07SCYRV5gInVzlcNlC2Pb7v5g46S5OCzNLivVD1pEK6XJSiDk8pLFd7z9jL2sXn/OFiQQKWZ3b0wIR4+qbJrezLQJYBE0RGAT28yuWxVZgkMIKDtl/2UCvk06SZzwj+JqGni491gKlbU1qyJvMattCC9/O11tcd/NzBb5Ugr+Kg3PoH+drpT0C2Y7DJwovaxTtIMBkXCd1wq4n1J4EVN/RVpkelhTf6N2sf9Rwcb1I7740xh4x//srh0yZpz4/sJP0XzWmLAks+e1EHWhTDsWG2Un+Dwtaa71KbxKrQSHKYD8TSJmxXrSCJ6KTRP0pGfLa7xDRbQM97LOEeF89Dshle1c= ceph-729b10fe-6f5a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:54:56.603 INFO:teuthology.orchestra.run.smithi162.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8IApLOVeOHtvaZle40hZ/i0UuxQbbFTC2FDoS2YjSGl5ARK5OjJMF8fQapT/Y9ffgfu/Aymmo3b77dv74EJ2EG4jIxeoAWmuJHs8ILxOU89jE58WBb1BpYx83q4IUwdiVotUAIi6JSOTG67NWOkerc6NZ2kMe9UbLC/KILR07SCYRV5gInVzlcNlC2Pb7v5g46S5OCzNLivVD1pEK6XJSiDk8pLFd7z9jL2sXn/OFiQQKWZ3b0wIR4+qbJrezLQJYBE0RGAT28yuWxVZgkMIKDtl/2UCvk06SZzwj+JqGni491gKlbU1qyJvMattCC9/O11tcd/NzBb5Ugr+Kg3PoH+drpT0C2Y7DJwovaxTtIMBkXCd1wq4n1J4EVN/RVpkelhTf6N2sf9Rwcb1I7740xh4x//srh0yZpz4/sJP0XzWmLAks+e1EHWhTDsWG2Un+Dwtaa71KbxKrQSHKYD8TSJmxXrSCJ6KTRP0pGfLa7xDRbQM97LOEeF89Dshle1c= ceph-729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:54:56.617 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T09:54:56.872 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:57.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.383+0000 7f4d672e6640 1 -- 172.21.15.45:0/4389850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 msgr2=0x7f4d601044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.383+0000 7f4d672e6640 1 --2- 172.21.15.45:0/4389850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d601044c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009880 tx=0x7f4d5c02a5d0 comp rx=0 tx=0).stop 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.383+0000 7f4d672e6640 1 -- 172.21.15.45:0/4389850 shutdown_connections 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.383+0000 7f4d672e6640 1 --2- 172.21.15.45:0/4389850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d601044c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.383+0000 7f4d672e6640 1 -- 172.21.15.45:0/4389850 >> 172.21.15.45:0/4389850 conn(0x7f4d600fbcd0 msgr2=0x7f4d600fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.384+0000 7f4d672e6640 1 -- 172.21.15.45:0/4389850 shutdown_connections 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.384+0000 7f4d672e6640 1 -- 172.21.15.45:0/4389850 wait complete. 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.384+0000 7f4d672e6640 1 Processor -- start 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.384+0000 7f4d672e6640 1 -- start start 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.384+0000 7f4d672e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d60196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.384+0000 7f4d672e6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60107670 con 0x7f4d601020d0 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.384+0000 7f4d65082640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d60196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d65082640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d60196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40702/0 (socket says 172.21.15.45:40702) 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d65082640 1 -- 172.21.15.45:0/2674929808 learned_addr learned my addr 172.21.15.45:0/2674929808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d65082640 1 -- 172.21.15.45:0/2674929808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d60197250 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d65082640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d60196d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c004f60 tx=0x7f4d5c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d527fc640 1 -- 172.21.15.45:0/2674929808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c03f070 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d527fc640 1 -- 172.21.15.45:0/2674929808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5c003ad0 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d60197540 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d527fc640 1 -- 172.21.15.45:0/2674929808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c035030 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.385+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6019a1c0 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.386+0000 7f4d527fc640 1 -- 172.21.15.45:0/2674929808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4d5c042070 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.386+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d60107df0 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.386+0000 7f4d527fc640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d4c039060 0x7f4d4c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.386+0000 7f4d527fc640 1 -- 172.21.15.45:0/2674929808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4d5c06cee0 con 0x7f4d601020d0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.387+0000 7f4d64881640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d4c039060 0x7f4d4c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.387+0000 7f4d64881640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d4c039060 0x7f4d4c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098c0 tx=0x7f4d54006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.391+0000 7f4d527fc640 1 -- 172.21.15.45:0/2674929808 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d5c039350 con 0x7f4d601020d0 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.570+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4d6002d8d0 con 0x7f4d601020d0 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.573+0000 7f4d527fc640 1 -- 172.21.15.45:0/2674929808 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4d5c003cf0 con 0x7f4d601020d0 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d4c039060 msgr2=0x7f4d4c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d4c039060 0x7f4d4c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098c0 tx=0x7f4d54006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 msgr2=0x7f4d60196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d60196d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c004f60 tx=0x7f4d5c0048f0 comp rx=0 tx=0).stop 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 shutdown_connections 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d4c039060 0x7f4d4c03b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 --2- 172.21.15.45:0/2674929808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d601020d0 0x7f4d60196d10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 >> 172.21.15.45:0/2674929808 conn(0x7f4d600fbcd0 msgr2=0x7f4d600fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 shutdown_connections 2024-09-10T09:54:57.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:57.576+0000 7f4d672e6640 1 -- 172.21.15.45:0/2674929808 wait complete. 2024-09-10T09:54:58.230 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T09:54:58.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T09:54:58.621 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:54:58 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2674929808' entity='client.admin' 2024-09-10T09:54:58.722 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.562+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3306333946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 msgr2=0x7f4cd41019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.562+0000 7f4cd9bcb640 1 --2- 172.21.15.45:0/3306333946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd41019e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009880 tx=0x7f4cc402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:54:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.562+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3306333946 shutdown_connections 2024-09-10T09:54:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.562+0000 7f4cd9bcb640 1 --2- 172.21.15.45:0/3306333946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd41019e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.562+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3306333946 >> 172.21.15.45:0/3306333946 conn(0x7f4cd40fcd70 msgr2=0x7f4cd40ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.562+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3306333946 shutdown_connections 2024-09-10T09:54:59.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.562+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3306333946 wait complete. 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.563+0000 7f4cd9bcb640 1 Processor -- start 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.563+0000 7f4cd9bcb640 1 -- start start 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd9bcb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd4196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd9bcb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41076f0 con 0x7f4cd41015e0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd4196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd4196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40724/0 (socket says 172.21.15.45:40724) 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd37fe640 1 -- 172.21.15.45:0/3139664994 learned_addr learned my addr 172.21.15.45:0/3139664994 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd37fe640 1 -- 172.21.15.45:0/3139664994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd41972a0 con 0x7f4cd41015e0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd37fe640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd4196d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4004f60 tx=0x7f4cc40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd0ff9640 1 -- 172.21.15.45:0/3139664994 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cc403f070 con 0x7f4cd41015e0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd4197530 con 0x7f4cd41015e0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd0ff9640 1 -- 172.21.15.45:0/3139664994 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cc4004d40 con 0x7f4cd41015e0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd0ff9640 1 -- 172.21.15.45:0/3139664994 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cc4035030 con 0x7f4cd41015e0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.564+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd419a220 con 0x7f4cd41015e0 2024-09-10T09:54:59.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.565+0000 7f4cd0ff9640 1 -- 172.21.15.45:0/3139664994 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4cc4042070 con 0x7f4cd41015e0 2024-09-10T09:54:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.566+0000 7f4cd0ff9640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4cbc039060 0x7f4cbc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.566+0000 7f4cd0ff9640 1 -- 172.21.15.45:0/3139664994 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4cc406cd50 con 0x7f4cd41015e0 2024-09-10T09:54:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.566+0000 7f4cd2ffd640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4cbc039060 0x7f4cbc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.566+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd4107e70 con 0x7f4cd41015e0 2024-09-10T09:54:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.566+0000 7f4cd2ffd640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4cbc039060 0x7f4cbc03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80098c0 tx=0x7f4cc8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:59.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.571+0000 7f4cd0ff9640 1 -- 172.21.15.45:0/3139664994 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cc4042360 con 0x7f4cd41015e0 2024-09-10T09:54:59.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.765+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f4cd4062630 con 0x7f4cbc039060 2024-09-10T09:54:59.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.770+0000 7f4cd0ff9640 1 -- 172.21.15.45:0/3139664994 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4cd4062630 con 0x7f4cbc039060 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4cbc039060 msgr2=0x7f4cbc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4cbc039060 0x7f4cbc03b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80098c0 tx=0x7f4cc8006db0 comp rx=0 tx=0).stop 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 msgr2=0x7f4cd4196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd4196d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4004f60 tx=0x7f4cc40048f0 comp rx=0 tx=0).stop 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 shutdown_connections 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4cbc039060 0x7f4cbc03b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 --2- 172.21.15.45:0/3139664994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4cd41015e0 0x7f4cd4196d60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:59.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 >> 172.21.15.45:0/3139664994 conn(0x7f4cd40fcd70 msgr2=0x7f4cd40fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 shutdown_connections 2024-09-10T09:54:59.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:54:59.772+0000 7f4cd9bcb640 1 -- 172.21.15.45:0/3139664994 wait complete. 2024-09-10T09:55:00.385 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi047 2024-09-10T09:55:00.386 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:55:00.386 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/etc/ceph/ceph.conf 2024-09-10T09:55:00.405 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:55:00.405 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:00.463 INFO:tasks.cephadm:Adding host smithi047 to orchestrator... 2024-09-10T09:55:00.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch host add smithi047 2024-09-10T09:55:01.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:00 smithi045 ceph-mon[31114]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:01.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:00 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:01.055 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.544+0000 7fc78e45d640 1 -- 172.21.15.45:0/3695103489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 msgr2=0x7fc788101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.544+0000 7fc78e45d640 1 --2- 172.21.15.45:0/3695103489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788101a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc778009880 tx=0x7fc77802a5d0 comp rx=0 tx=0).stop 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.544+0000 7fc78e45d640 1 -- 172.21.15.45:0/3695103489 shutdown_connections 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.544+0000 7fc78e45d640 1 --2- 172.21.15.45:0/3695103489 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788101a20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.544+0000 7fc78e45d640 1 -- 172.21.15.45:0/3695103489 >> 172.21.15.45:0/3695103489 conn(0x7fc7880fcdf0 msgr2=0x7fc7880ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.544+0000 7fc78e45d640 1 -- 172.21.15.45:0/3695103489 shutdown_connections 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.544+0000 7fc78e45d640 1 -- 172.21.15.45:0/3695103489 wait complete. 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc78e45d640 1 Processor -- start 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc78e45d640 1 -- start start 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc78e45d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc78e45d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc788107730 con 0x7fc788101620 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc787fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc787fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40744/0 (socket says 172.21.15.45:40744) 2024-09-10T09:55:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc787fff640 1 -- 172.21.15.45:0/2802452940 learned_addr learned my addr 172.21.15.45:0/2802452940 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.545+0000 7fc787fff640 1 -- 172.21.15.45:0/2802452940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7880763f0 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.546+0000 7fc787fff640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788075eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc778004f60 tx=0x7fc7780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.546+0000 7fc7857fa640 1 -- 172.21.15.45:0/2802452940 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc77803f070 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.546+0000 7fc7857fa640 1 -- 172.21.15.45:0/2802452940 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc778004cc0 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.546+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7880745e0 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.546+0000 7fc7857fa640 1 -- 172.21.15.45:0/2802452940 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc778035030 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.546+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc788074b20 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.547+0000 7fc7857fa640 1 -- 172.21.15.45:0/2802452940 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc778042070 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.547+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc788107eb0 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.547+0000 7fc7857fa640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc7700390c0 0x7fc77003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.547+0000 7fc7857fa640 1 -- 172.21.15.45:0/2802452940 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc77806ce40 con 0x7fc788101620 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.547+0000 7fc7877fe640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc7700390c0 0x7fc77003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:01.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.548+0000 7fc7877fe640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc7700390c0 0x7fc77003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0098c0 tx=0x7fc77c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:01.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.552+0000 7fc7857fa640 1 -- 172.21.15.45:0/2802452940 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7780433b0 con 0x7fc788101620 2024-09-10T09:55:01.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:01.734+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "target": ["mon-mgr", ""]}) -- 0x7fc7881901f0 con 0x7fc7700390c0 2024-09-10T09:55:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-10T09:55:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:02.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:02.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:02.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:02.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:02.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:02.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:02.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:03.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:03 smithi045 ceph-mon[31114]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:03.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:03 smithi045 ceph-mon[31114]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:55:03.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:03 smithi045 ceph-mon[31114]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:04.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:04 smithi045 ceph-mon[31114]: Deploying cephadm binary to smithi047 2024-09-10T09:55:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.602+0000 7fc7857fa640 1 -- 172.21.15.45:0/2802452940 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc7881901f0 con 0x7fc7700390c0 2024-09-10T09:55:05.604 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi047' with addr '172.21.15.47' 2024-09-10T09:55:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc7700390c0 msgr2=0x7fc77003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc7700390c0 0x7fc77003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0098c0 tx=0x7fc77c006db0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 msgr2=0x7fc788075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788075eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc778004f60 tx=0x7fc7780048f0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 shutdown_connections 2024-09-10T09:55:05.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc7700390c0 0x7fc77003b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 --2- 172.21.15.45:0/2802452940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc788101620 0x7fc788075eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 >> 172.21.15.45:0/2802452940 conn(0x7fc7880fcdf0 msgr2=0x7fc7880fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 shutdown_connections 2024-09-10T09:55:05.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:05.605+0000 7fc78e45d640 1 -- 172.21.15.45:0/2802452940 wait complete. 2024-09-10T09:55:06.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T09:55:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:06.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:06.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:06 smithi045 ceph-mon[31114]: Added host smithi047 2024-09-10T09:55:06.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:06.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:06.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.902+0000 7f918d316640 1 -- 172.21.15.45:0/879695341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 msgr2=0x7f9188100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.902+0000 7f918d316640 1 --2- 172.21.15.45:0/879695341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f9188100920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9178009880 tx=0x7f917802a670 comp rx=0 tx=0).stop 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.902+0000 7f918d316640 1 -- 172.21.15.45:0/879695341 shutdown_connections 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.902+0000 7f918d316640 1 --2- 172.21.15.45:0/879695341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f9188100920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.902+0000 7f918d316640 1 -- 172.21.15.45:0/879695341 >> 172.21.15.45:0/879695341 conn(0x7f91880fc1b0 msgr2=0x7f91880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.902+0000 7f918d316640 1 -- 172.21.15.45:0/879695341 shutdown_connections 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.902+0000 7f918d316640 1 -- 172.21.15.45:0/879695341 wait complete. 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.903+0000 7f918d316640 1 Processor -- start 2024-09-10T09:55:06.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.903+0000 7f918d316640 1 -- start start 2024-09-10T09:55:06.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.903+0000 7f918d316640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f91881a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.903+0000 7f918d316640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91881122d0 con 0x7f9188100540 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.903+0000 7f9186ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f91881a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.903+0000 7f9186ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f91881a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49664/0 (socket says 172.21.15.45:49664) 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.903+0000 7f9186ffd640 1 -- 172.21.15.45:0/4276284432 learned_addr learned my addr 172.21.15.45:0/4276284432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.904+0000 7f9186ffd640 1 -- 172.21.15.45:0/4276284432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91881a3980 con 0x7f9188100540 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.904+0000 7f9186ffd640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f91881a3440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9178009230 tx=0x7f9178003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.904+0000 7f9184ff9640 1 -- 172.21.15.45:0/4276284432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f917803f070 con 0x7f9188100540 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.904+0000 7f9184ff9640 1 -- 172.21.15.45:0/4276284432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917803bc80 con 0x7f9188100540 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.904+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91881a3c10 con 0x7f9188100540 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.904+0000 7f9184ff9640 1 -- 172.21.15.45:0/4276284432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9178035030 con 0x7f9188100540 2024-09-10T09:55:06.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.904+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918819d600 con 0x7f9188100540 2024-09-10T09:55:06.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.905+0000 7f9184ff9640 1 -- 172.21.15.45:0/4276284432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9178042070 con 0x7f9188100540 2024-09-10T09:55:06.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.905+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918806a7b0 con 0x7f9188100540 2024-09-10T09:55:06.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.905+0000 7f9184ff9640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f91740390c0 0x7f917403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:06.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.905+0000 7f9184ff9640 1 -- 172.21.15.45:0/4276284432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f917806cd40 con 0x7f9188100540 2024-09-10T09:55:06.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.905+0000 7f917e7fc640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f91740390c0 0x7f917403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:06.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.906+0000 7f917e7fc640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f91740390c0 0x7f917403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91700098c0 tx=0x7f9170006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:06.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:06.910+0000 7f9184ff9640 1 -- 172.21.15.45:0/4276284432 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91780314c0 con 0x7f9188100540 2024-09-10T09:55:07.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.090+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f91881009a0 con 0x7f91740390c0 2024-09-10T09:55:07.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.091+0000 7f9184ff9640 1 -- 172.21.15.45:0/4276284432 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f91881009a0 con 0x7f91740390c0 2024-09-10T09:55:07.092 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:55:07.092 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}] 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f91740390c0 msgr2=0x7f917403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f91740390c0 0x7f917403b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f91700098c0 tx=0x7f9170006db0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 msgr2=0x7f91881a3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f91881a3440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9178009230 tx=0x7f9178003d40 comp rx=0 tx=0).stop 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 shutdown_connections 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f91740390c0 0x7f917403b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 --2- 172.21.15.45:0/4276284432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9188100540 0x7f91881a3440 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 >> 172.21.15.45:0/4276284432 conn(0x7f91880fc1b0 msgr2=0x7f91880fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:07.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 shutdown_connections 2024-09-10T09:55:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:07.093+0000 7f918d316640 1 -- 172.21.15.45:0/4276284432 wait complete. 2024-09-10T09:55:07.731 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi162 2024-09-10T09:55:07.731 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:55:07.731 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.conf 2024-09-10T09:55:07.752 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:55:07.752 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:07.810 INFO:tasks.cephadm:Adding host smithi162 to orchestrator... 2024-09-10T09:55:07.810 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch host add smithi162 2024-09-10T09:55:08.067 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:08.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:08.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:07 smithi045 ceph-mon[31114]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:08.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.495+0000 7f03a09fa640 1 -- 172.21.15.45:0/2912729303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 msgr2=0x7f039c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.495+0000 7f03a09fa640 1 --2- 172.21.15.45:0/2912729303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c106920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0384009810 tx=0x7f038402a540 comp rx=0 tx=0).stop 2024-09-10T09:55:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.495+0000 7f03a09fa640 1 -- 172.21.15.45:0/2912729303 shutdown_connections 2024-09-10T09:55:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.495+0000 7f03a09fa640 1 --2- 172.21.15.45:0/2912729303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c106920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.495+0000 7f03a09fa640 1 -- 172.21.15.45:0/2912729303 >> 172.21.15.45:0/2912729303 conn(0x7f039c077310 msgr2=0x7f039c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.495+0000 7f03a09fa640 1 -- 172.21.15.45:0/2912729303 shutdown_connections 2024-09-10T09:55:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.495+0000 7f03a09fa640 1 -- 172.21.15.45:0/2912729303 wait complete. 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.496+0000 7f03a09fa640 1 Processor -- start 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.496+0000 7f03a09fa640 1 -- start start 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.496+0000 7f03a09fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.496+0000 7f03a09fa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f039c10ab70 con 0x7f039c106540 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f039a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f039a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49682/0 (socket says 172.21.15.45:49682) 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f039a59c640 1 -- 172.21.15.45:0/4185486260 learned_addr learned my addr 172.21.15.45:0/4185486260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:55:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f039a59c640 1 -- 172.21.15.45:0/4185486260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c197600 con 0x7f039c106540 2024-09-10T09:55:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f039a59c640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c1970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f03840045e0 tx=0x7f0384004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f03937fe640 1 -- 172.21.15.45:0/4185486260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0384030e70 con 0x7f039c106540 2024-09-10T09:55:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f03937fe640 1 -- 172.21.15.45:0/4185486260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03840318e0 con 0x7f039c106540 2024-09-10T09:55:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c197890 con 0x7f039c106540 2024-09-10T09:55:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f03937fe640 1 -- 172.21.15.45:0/4185486260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03840049c0 con 0x7f039c106540 2024-09-10T09:55:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.497+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c19b990 con 0x7f039c106540 2024-09-10T09:55:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.498+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c101bb0 con 0x7f039c106540 2024-09-10T09:55:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.498+0000 7f03937fe640 1 -- 172.21.15.45:0/4185486260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0384039040 con 0x7f039c106540 2024-09-10T09:55:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.499+0000 7f03937fe640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0388039060 0x7f038803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.499+0000 7f03937fe640 1 -- 172.21.15.45:0/4185486260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f038406d290 con 0x7f039c106540 2024-09-10T09:55:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.499+0000 7f0399d9b640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0388039060 0x7f038803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.499+0000 7f0399d9b640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0388039060 0x7f038803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f038c0098c0 tx=0x7f038c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.504+0000 7f03937fe640 1 -- 172.21.15.45:0/4185486260 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0384040030 con 0x7f039c106540 2024-09-10T09:55:08.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:08.686+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}) -- 0x7f039c1069a0 con 0x7f0388039060 2024-09-10T09:55:09.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:09 smithi045 ceph-mon[31114]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:09.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:09 smithi045 ceph-mon[31114]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: Deploying cephadm binary to smithi162 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:12.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:11 smithi045 ceph-mon[31114]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.593+0000 7f03937fe640 1 -- 172.21.15.45:0/4185486260 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f039c1069a0 con 0x7f0388039060 2024-09-10T09:55:12.595 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi162' with addr '172.21.15.162' 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0388039060 msgr2=0x7f038803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0388039060 0x7f038803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f038c0098c0 tx=0x7f038c006db0 comp rx=0 tx=0).stop 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 msgr2=0x7f039c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c1970c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f03840045e0 tx=0x7f0384004610 comp rx=0 tx=0).stop 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 shutdown_connections 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0388039060 0x7f038803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 --2- 172.21.15.45:0/4185486260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c106540 0x7f039c1970c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 >> 172.21.15.45:0/4185486260 conn(0x7f039c077310 msgr2=0x7f039c0696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 shutdown_connections 2024-09-10T09:55:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:12.596+0000 7f03a09fa640 1 -- 172.21.15.45:0/4185486260 wait complete. 2024-09-10T09:55:13.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T09:55:13.437 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:13.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:13 smithi045 ceph-mon[31114]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:13.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:13.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:13 smithi045 ceph-mon[31114]: Added host smithi162 2024-09-10T09:55:13.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:13.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:13.816 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:13.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.899+0000 7f70ee99d640 1 -- 172.21.15.45:0/3148318080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 msgr2=0x7f70e8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.899+0000 7f70ee99d640 1 --2- 172.21.15.45:0/3148318080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009880 tx=0x7f70d802a5d0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.899+0000 7f70ee99d640 1 -- 172.21.15.45:0/3148318080 shutdown_connections 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.899+0000 7f70ee99d640 1 --2- 172.21.15.45:0/3148318080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.899+0000 7f70ee99d640 1 -- 172.21.15.45:0/3148318080 >> 172.21.15.45:0/3148318080 conn(0x7f70e80fc1b0 msgr2=0x7f70e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.899+0000 7f70ee99d640 1 -- 172.21.15.45:0/3148318080 shutdown_connections 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.899+0000 7f70ee99d640 1 -- 172.21.15.45:0/3148318080 wait complete. 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.900+0000 7f70ee99d640 1 Processor -- start 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.900+0000 7f70ee99d640 1 -- start start 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.900+0000 7f70ee99d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.900+0000 7f70ee99d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70e810ab90 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.900+0000 7f70e7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.900+0000 7f70e7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44218/0 (socket says 172.21.15.45:44218) 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.900+0000 7f70e7fff640 1 -- 172.21.15.45:0/2455829089 learned_addr learned my addr 172.21.15.45:0/2455829089 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.901+0000 7f70e7fff640 1 -- 172.21.15.45:0/2455829089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e81984e0 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.901+0000 7f70e7fff640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8197fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009230 tx=0x7f70d8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.901+0000 7f70e5ffb640 1 -- 172.21.15.45:0/2455829089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d803f070 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.901+0000 7f70e5ffb640 1 -- 172.21.15.45:0/2455829089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70d803bc80 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.901+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e8198770 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.901+0000 7f70e5ffb640 1 -- 172.21.15.45:0/2455829089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70d8035030 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.901+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e8198bd0 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.902+0000 7f70e5ffb640 1 -- 172.21.15.45:0/2455829089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f70d8042070 con 0x7f70e8106560 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.902+0000 7f70e5ffb640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f70d4039060 0x7f70d403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.902+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e8101bd0 con 0x7f70e8106560 2024-09-10T09:55:13.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.902+0000 7f70e5ffb640 1 -- 172.21.15.45:0/2455829089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f70d806cd60 con 0x7f70e8106560 2024-09-10T09:55:13.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.902+0000 7f70dffff640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f70d4039060 0x7f70d403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.903+0000 7f70dffff640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f70d4039060 0x7f70d403b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f70d00098c0 tx=0x7f70d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:13.907+0000 7f70e5ffb640 1 -- 172.21.15.45:0/2455829089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70d8043c30 con 0x7f70e8106560 2024-09-10T09:55:14.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.086+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70e81069c0 con 0x7f70d4039060 2024-09-10T09:55:14.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.087+0000 7f70e5ffb640 1 -- 172.21.15.45:0/2455829089 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f70e81069c0 con 0x7f70d4039060 2024-09-10T09:55:14.088 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:55:14.088 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.162", "hostname": "smithi162", "labels": [], "status": ""}] 2024-09-10T09:55:14.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f70d4039060 msgr2=0x7f70d403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f70d4039060 0x7f70d403b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f70d00098c0 tx=0x7f70d0006db0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 msgr2=0x7f70e8197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8197fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009230 tx=0x7f70d8003d40 comp rx=0 tx=0).stop 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 shutdown_connections 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f70d4039060 0x7f70d403b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 --2- 172.21.15.45:0/2455829089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70e8106560 0x7f70e8197fa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 >> 172.21.15.45:0/2455829089 conn(0x7f70e80fc1b0 msgr2=0x7f70e80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 shutdown_connections 2024-09-10T09:55:14.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:14.089+0000 7f70ee99d640 1 -- 172.21.15.45:0/2455829089 wait complete. 2024-09-10T09:55:14.668 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T09:55:14.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T09:55:14.923 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:15.264 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:15.264 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:14 smithi045 ceph-mon[31114]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.349+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3775886477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 msgr2=0x7fb0d8107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.349+0000 7fb0dd45e640 1 --2- 172.21.15.45:0/3775886477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d8107a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009880 tx=0x7fb0c802a5d0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.349+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3775886477 shutdown_connections 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.349+0000 7fb0dd45e640 1 --2- 172.21.15.45:0/3775886477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d8107a50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.349+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3775886477 >> 172.21.15.45:0/3775886477 conn(0x7fb0d80fd260 msgr2=0x7fb0d80ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.349+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3775886477 shutdown_connections 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.349+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3775886477 wait complete. 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.350+0000 7fb0dd45e640 1 Processor -- start 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.350+0000 7fb0dd45e640 1 -- start start 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.350+0000 7fb0dd45e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.350+0000 7fb0dd45e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d810bca0 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.350+0000 7fb0d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.350+0000 7fb0d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44234/0 (socket says 172.21.15.45:44234) 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.350+0000 7fb0d6ffd640 1 -- 172.21.15.45:0/3251812678 learned_addr learned my addr 172.21.15.45:0/3251812678 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.351+0000 7fb0d6ffd640 1 -- 172.21.15.45:0/3251812678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d80763e0 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.351+0000 7fb0d6ffd640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d80744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009230 tx=0x7fb0c8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.351+0000 7fb0bffff640 1 -- 172.21.15.45:0/3251812678 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c803f070 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.351+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d8074a30 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.351+0000 7fb0bffff640 1 -- 172.21.15.45:0/3251812678 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c803bd40 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.351+0000 7fb0bffff640 1 -- 172.21.15.45:0/3251812678 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c8035030 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.351+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d8074ef0 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.352+0000 7fb0bffff640 1 -- 172.21.15.45:0/3251812678 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb0c8042070 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.352+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d8102ce0 con 0x7fb0d8107670 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.352+0000 7fb0bffff640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb0c0039060 0x7fb0c003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.352+0000 7fb0bffff640 1 -- 172.21.15.45:0/3251812678 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb0c806dd00 con 0x7fb0d8107670 2024-09-10T09:55:15.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.352+0000 7fb0d67fc640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb0c0039060 0x7fb0c003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.353+0000 7fb0d67fc640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb0c0039060 0x7fb0c003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0098c0 tx=0x7fb0cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.357+0000 7fb0bffff640 1 -- 172.21.15.45:0/3251812678 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0c8031de0 con 0x7fb0d8107670 2024-09-10T09:55:15.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.536+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb0d8075a20 con 0x7fb0d8107670 2024-09-10T09:55:15.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.952+0000 7fb0bffff640 1 -- 172.21.15.45:0/3251812678 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fb0c806c610 con 0x7fb0d8107670 2024-09-10T09:55:15.954 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb0c0039060 msgr2=0x7fb0c003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb0c0039060 0x7fb0c003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0098c0 tx=0x7fb0cc006db0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 msgr2=0x7fb0d80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d80744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009230 tx=0x7fb0c8003d80 comp rx=0 tx=0).stop 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 shutdown_connections 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb0c0039060 0x7fb0c003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 --2- 172.21.15.45:0/3251812678 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb0d8107670 0x7fb0d80744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.954+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 >> 172.21.15.45:0/3251812678 conn(0x7fb0d80fd260 msgr2=0x7fb0d80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.955+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 shutdown_connections 2024-09-10T09:55:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:15.955+0000 7fb0dd45e640 1 -- 172.21.15.45:0/3251812678 wait complete. 2024-09-10T09:55:16.268 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:15 smithi045 ceph-mon[31114]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:16.268 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:15 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3251812678' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T09:55:16.562 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-09-10T09:55:16.563 INFO:tasks.cephadm:Adding mon.b on smithi047 2024-09-10T09:55:16.563 INFO:tasks.cephadm:Adding mon.c on smithi162 2024-09-10T09:55:16.563 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi047:172.21.15.47=b;smithi162:172.21.15.162=c' 2024-09-10T09:55:16.869 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T09:55:16.913 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3251812678' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:17.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.458+0000 7f52b6f1b640 1 -- 172.21.15.162:0/309510405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 msgr2=0x7f52b0104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:17.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.458+0000 7f52b6f1b640 1 --2- 172.21.15.162:0/309510405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0104490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009c20 tx=0x7f52ac02aa90 comp rx=0 tx=0).stop 2024-09-10T09:55:17.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.458+0000 7f52b6f1b640 1 -- 172.21.15.162:0/309510405 shutdown_connections 2024-09-10T09:55:17.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.458+0000 7f52b6f1b640 1 --2- 172.21.15.162:0/309510405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0104490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.458+0000 7f52b6f1b640 1 -- 172.21.15.162:0/309510405 >> 172.21.15.162:0/309510405 conn(0x7f52b00fbca0 msgr2=0x7f52b00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:17.483 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.458+0000 7f52b6f1b640 1 -- 172.21.15.162:0/309510405 shutdown_connections 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.458+0000 7f52b6f1b640 1 -- 172.21.15.162:0/309510405 wait complete. 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.459+0000 7f52b6f1b640 1 Processor -- start 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.459+0000 7f52b6f1b640 1 -- start start 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.459+0000 7f52b6f1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.459+0000 7f52b6f1b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01076b0 con 0x7f52b01020a0 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.459+0000 7f52b4cb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52b4cb7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.162:47874/0 (socket says 172.21.15.162:47874) 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52b4cb7640 1 -- 172.21.15.162:0/3204673368 learned_addr learned my addr 172.21.15.162:0/3204673368 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52b4cb7640 1 -- 172.21.15.162:0/3204673368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b0197240 con 0x7f52b01020a0 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52b4cb7640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0196d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f52ac006010 tx=0x7f52ac02e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52a5ffb640 1 -- 172.21.15.162:0/3204673368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ac047070 con 0x7f52b01020a0 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52a5ffb640 1 -- 172.21.15.162:0/3204673368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac003e50 con 0x7f52b01020a0 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b0197530 con 0x7f52b01020a0 2024-09-10T09:55:17.484 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.460+0000 7f52a5ffb640 1 -- 172.21.15.162:0/3204673368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52ac033050 con 0x7f52b01020a0 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.461+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b019a1b0 con 0x7f52b01020a0 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.461+0000 7f52a5ffb640 1 -- 172.21.15.162:0/3204673368 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f52ac04a070 con 0x7f52b01020a0 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.461+0000 7f52a5ffb640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5298039060 0x7f529803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.462+0000 7f52a5ffb640 1 -- 172.21.15.162:0/3204673368 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f52ac03e030 con 0x7f52b01020a0 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.462+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b0107e30 con 0x7f52b01020a0 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.462+0000 7f52a7fff640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5298039060 0x7f529803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.462+0000 7f52a7fff640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5298039060 0x7f529803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52a00098c0 tx=0x7f52a0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:17.485 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.467+0000 7f52a5ffb640 1 -- 172.21.15.162:0/3204673368 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52ac051020 con 0x7f52b01020a0 2024-09-10T09:55:17.654 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.652+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi047:172.21.15.47=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}) -- 0x7f52b002d960 con 0x7f5298039060 2024-09-10T09:55:17.660 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.658+0000 7f52a5ffb640 1 -- 172.21.15.162:0/3204673368 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f52b002d960 con 0x7f5298039060 2024-09-10T09:55:17.660 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mon update... 2024-09-10T09:55:17.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5298039060 msgr2=0x7f529803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:17.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5298039060 0x7f529803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f52a00098c0 tx=0x7f52a0006db0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 msgr2=0x7f52b0196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:17.664 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0196d00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f52ac006010 tx=0x7f52ac02e3c0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 shutdown_connections 2024-09-10T09:55:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5298039060 0x7f529803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 --2- 172.21.15.162:0/3204673368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52b01020a0 0x7f52b0196d00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 >> 172.21.15.162:0/3204673368 conn(0x7f52b00fbca0 msgr2=0x7f52b00fc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 shutdown_connections 2024-09-10T09:55:17.665 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:17.660+0000 7f52b6f1b640 1 -- 172.21.15.162:0/3204673368 wait complete. 2024-09-10T09:55:18.393 DEBUG:teuthology.orchestra.run.smithi047:mon.b> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T09:55:18.396 DEBUG:teuthology.orchestra.run.smithi162:mon.c> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T09:55:18.399 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:55:18.399 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:18.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:18.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:18.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:18.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:18.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:18.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:18.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:18 smithi045 ceph-mon[31114]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi047:172.21.15.47=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:18 smithi045 ceph-mon[31114]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi047:172.21.15.47=b;smithi162:172.21.15.162=c;count:3 2024-09-10T09:55:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:18 smithi045 ceph-mon[31114]: Deploying daemon mon.c on smithi162 2024-09-10T09:55:19.546 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:20.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:19 smithi045 ceph-mon[31114]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:20.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 -- 172.21.15.162:0/782021029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 msgr2=0x7f8308102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 --2- 172.21.15.162:0/782021029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f8308102b10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009810 tx=0x7f82f802a540 comp rx=0 tx=0).stop 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 -- 172.21.15.162:0/782021029 shutdown_connections 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 --2- 172.21.15.162:0/782021029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f8308102b10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 -- 172.21.15.162:0/782021029 >> 172.21.15.162:0/782021029 conn(0x7f8308077310 msgr2=0x7f83080fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 -- 172.21.15.162:0/782021029 shutdown_connections 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 -- 172.21.15.162:0/782021029 wait complete. 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 Processor -- start 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.488+0000 7f830e012640 1 -- start start 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.489+0000 7f830e012640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f83081971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:20.491 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.489+0000 7f830e012640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f830810bc80 con 0x7f8308102730 2024-09-10T09:55:20.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.489+0000 7f83077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f83081971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.489+0000 7f83077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f83081971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.162:34896/0 (socket says 172.21.15.162:34896) 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.489+0000 7f83077fe640 1 -- 172.21.15.162:0/3919653945 learned_addr learned my addr 172.21.15.162:0/3919653945 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.489+0000 7f83077fe640 1 -- 172.21.15.162:0/3919653945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83081976f0 con 0x7f8308102730 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.489+0000 7f83077fe640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f83081971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82f80045e0 tx=0x7f82f8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.490+0000 7f83057fa640 1 -- 172.21.15.162:0/3919653945 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82f8030e70 con 0x7f8308102730 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.490+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8308197980 con 0x7f8308102730 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.490+0000 7f83057fa640 1 -- 172.21.15.162:0/3919653945 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f80318e0 con 0x7f8308102730 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.490+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8308197de0 con 0x7f8308102730 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.490+0000 7f83057fa640 1 -- 172.21.15.162:0/3919653945 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82f80049c0 con 0x7f8308102730 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.491+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8308069f90 con 0x7f8308102730 2024-09-10T09:55:20.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.491+0000 7f83057fa640 1 -- 172.21.15.162:0/3919653945 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f82f8039040 con 0x7f8308102730 2024-09-10T09:55:20.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.491+0000 7f83057fa640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f82f4039060 0x7f82f403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:20.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.491+0000 7f83057fa640 1 -- 172.21.15.162:0/3919653945 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f82f806d5a0 con 0x7f8308102730 2024-09-10T09:55:20.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.491+0000 7f82feffd640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f82f4039060 0x7f82f403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:20.494 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.492+0000 7f82feffd640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f82f4039060 0x7f82f403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f82f00098c0 tx=0x7f82f0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:20.498 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.496+0000 7f83057fa640 1 -- 172.21.15.162:0/3919653945 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82f8040030 con 0x7f8308102730 2024-09-10T09:55:20.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.740+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8308198570 con 0x7f8308102730 2024-09-10T09:55:20.742 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.740+0000 7f83057fa640 1 -- 172.21.15.162:0/3919653945 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f82f8037d70 con 0x7f8308102730 2024-09-10T09:55:20.743 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:55:20.743 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":1,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","modified":"2024-09-10T09:53:50.319028Z","created":"2024-09-10T09:53:50.319028Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T09:55:20.743 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 1 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f82f4039060 msgr2=0x7f82f403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f82f4039060 0x7f82f403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f82f00098c0 tx=0x7f82f0006db0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 msgr2=0x7f83081971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f83081971b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82f80045e0 tx=0x7f82f8004610 comp rx=0 tx=0).stop 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 shutdown_connections 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f82f4039060 0x7f82f403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 --2- 172.21.15.162:0/3919653945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8308102730 0x7f83081971b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 >> 172.21.15.162:0/3919653945 conn(0x7f8308077310 msgr2=0x7f83081054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:20.745 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 shutdown_connections 2024-09-10T09:55:20.746 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:20.743+0000 7f830e012640 1 -- 172.21.15.162:0/3919653945 wait complete. 2024-09-10T09:55:21.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:20 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/3919653945' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:22.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:21 smithi045 ceph-mon[31114]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:22.414 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.414 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.414 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.414 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:22.414 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:22.414 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:22.415 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi047:172.21.15.47=b;smithi162:172.21.15.162=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi047:172.21.15.47=b;smithi162:172.21.15.162=c;count:3 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: Deploying daemon mon.c on smithi162 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/3919653945' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:22.416 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 ceph-mon[34963]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T09:55:22.552 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:55:22.553 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:55:22.799 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:55:22.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 bash[34944]: ca3816f4365a47fceb39b33020678df704a1218600f64f044f1a9913b5f68c29 2024-09-10T09:55:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:22 smithi162 systemd[1]: Started Ceph mon.c for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T09:55:27.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.590+0000 7fcf951f2640 1 -- 172.21.15.162:0/1080011401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003530 msgr2=0x7fcf780059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.590+0000 7fcf951f2640 1 --2- 172.21.15.162:0/1080011401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003530 0x7fcf780059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf90102af0 tx=0x7fcf8002c490 comp rx=0 tx=0).stop 2024-09-10T09:55:27.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.590+0000 7fcf951f2640 1 -- 172.21.15.162:0/1080011401 shutdown_connections 2024-09-10T09:55:27.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.590+0000 7fcf951f2640 1 --2- 172.21.15.162:0/1080011401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf78003530 0x7fcf780059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.590+0000 7fcf951f2640 1 -- 172.21.15.162:0/1080011401 >> 172.21.15.162:0/1080011401 conn(0x7fcf900fd320 msgr2=0x7fcf900ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.590+0000 7fcf951f2640 1 -- 172.21.15.162:0/1080011401 shutdown_connections 2024-09-10T09:55:27.592 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.590+0000 7fcf951f2640 1 -- 172.21.15.162:0/1080011401 wait complete. 2024-09-10T09:55:27.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.591+0000 7fcf951f2640 1 Processor -- start 2024-09-10T09:55:27.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf951f2640 1 -- start start 2024-09-10T09:55:27.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf951f2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 0x7fcf901972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf951f2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf901977e0 0x7fcf9019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf951f2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90109780 con 0x7fcf901977e0 2024-09-10T09:55:27.593 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf951f2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90109600 con 0x7fcf78003530 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf8ed9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 0x7fcf901972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf8ed9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 0x7fcf901972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:53202/0 (socket says 172.21.15.162:53202) 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.592+0000 7fcf8ed9d640 1 -- 172.21.15.162:0/2494614159 learned_addr learned my addr 172.21.15.162:0/2494614159 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8ed9d640 1 -- 172.21.15.162:0/2494614159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 msgr2=0x7fcf901972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8ed9d640 1 -- 172.21.15.162:0/2494614159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 msgr2=0x7fcf901972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8ed9d640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 0x7fcf901972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8ed9d640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 0x7fcf901972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8e59c640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf901977e0 0x7fcf9019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8e59c640 1 -- 172.21.15.162:0/2494614159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 msgr2=0x7fcf901972a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8e59c640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 0x7fcf901972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8e59c640 1 -- 172.21.15.162:0/2494614159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90197e00 con 0x7fcf901977e0 2024-09-10T09:55:27.595 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf8e59c640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf901977e0 0x7fcf9019bb70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400b3d0 tx=0x7fcf8400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:27.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf77fff640 1 -- 172.21.15.162:0/2494614159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcf84011630 con 0x7fcf901977e0 2024-09-10T09:55:27.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf77fff640 1 -- 172.21.15.162:0/2494614159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf84011cb0 con 0x7fcf901977e0 2024-09-10T09:55:27.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf77fff640 1 -- 172.21.15.162:0/2494614159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcf84010df0 con 0x7fcf901977e0 2024-09-10T09:55:27.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.593+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9019c0b0 con 0x7fcf901977e0 2024-09-10T09:55:27.596 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.594+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9019c5f0 con 0x7fcf901977e0 2024-09-10T09:55:27.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.594+0000 7fcf77fff640 1 -- 172.21.15.162:0/2494614159 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcf84018020 con 0x7fcf901977e0 2024-09-10T09:55:27.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.595+0000 7fcf77fff640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcf7c038e00 0x7fcf7c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.595+0000 7fcf77fff640 1 -- 172.21.15.162:0/2494614159 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcf84010510 con 0x7fcf901977e0 2024-09-10T09:55:27.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.595+0000 7fcf8ed9d640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcf7c038e00 0x7fcf7c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.595+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf90102da0 con 0x7fcf901977e0 2024-09-10T09:55:27.597 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.595+0000 7fcf8ed9d640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcf7c038e00 0x7fcf7c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf80012c10 tx=0x7fcf800323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:27.601 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.600+0000 7fcf77fff640 1 -- 172.21.15.162:0/2494614159 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf8401f020 con 0x7fcf901977e0 2024-09-10T09:55:27.837 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.835+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcf9019c930 con 0x7fcf901977e0 2024-09-10T09:55:27.837 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:55:27.837 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: mon.a calling monitor election 2024-09-10T09:55:27.837 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: mon.c calling monitor election 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: monmap epoch 2 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: last_changed 2024-09-10T09:55:22.418058+0000 2024-09-10T09:55:27.838 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: min_mon_release 19 (squid) 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: election_strategy: 1 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: fsmap 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: mgrmap e13: x(active, since 38s) 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: overall HEALTH_OK 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:27.839 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:27 smithi162 ceph-mon[34963]: Deploying daemon mon.b on smithi047 2024-09-10T09:55:27.840 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.836+0000 7fcf77fff640 1 -- 172.21.15.162:0/2494614159 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fcf84018310 con 0x7fcf901977e0 2024-09-10T09:55:27.840 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:55:27.840 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":2,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","modified":"2024-09-10T09:55:22.418058Z","created":"2024-09-10T09:53:50.319028Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 2 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcf7c038e00 msgr2=0x7fcf7c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcf7c038e00 0x7fcf7c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcf80012c10 tx=0x7fcf800323a0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf901977e0 msgr2=0x7fcf9019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf901977e0 0x7fcf9019bb70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400b3d0 tx=0x7fcf8400b890 comp rx=0 tx=0).stop 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 shutdown_connections 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fcf7c038e00 0x7fcf7c03b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf901977e0 0x7fcf9019bb70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 --2- 172.21.15.162:0/2494614159 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcf78003530 0x7fcf901972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 >> 172.21.15.162:0/2494614159 conn(0x7fcf900fd320 msgr2=0x7fcf900ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 shutdown_connections 2024-09-10T09:55:27.841 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:27.839+0000 7fcf951f2640 1 -- 172.21.15.162:0/2494614159 wait complete. 2024-09-10T09:55:27.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:55:27.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: mon.a calling monitor election 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: mon.c calling monitor election 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:55:27.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: monmap epoch 2 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: last_changed 2024-09-10T09:55:22.418058+0000 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: min_mon_release 19 (squid) 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: election_strategy: 1 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: fsmap 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: mgrmap e13: x(active, since 38s) 2024-09-10T09:55:27.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: overall HEALTH_OK 2024-09-10T09:55:27.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:27.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:27.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:27.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:27.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:27.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:27 smithi045 ceph-mon[31114]: Deploying daemon mon.b on smithi047 2024-09-10T09:55:28.576 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:28 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/2494614159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:28.576 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:28.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:28 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/2494614159' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:28.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:29.605 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:55:29.605 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:55:29.860 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:55:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:29 smithi162 ceph-mon[34963]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:29.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:29 smithi045 ceph-mon[31114]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.251+0000 7f9907963640 1 -- 172.21.15.162:0/2254741542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 msgr2=0x7f9900107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.251+0000 7f9907963640 1 --2- 172.21.15.162:0/2254741542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 0x7f9900107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009880 tx=0x7f98f402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.251+0000 7f9907963640 1 -- 172.21.15.162:0/2254741542 shutdown_connections 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.251+0000 7f9907963640 1 --2- 172.21.15.162:0/2254741542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 0x7f9900107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.251+0000 7f9907963640 1 -- 172.21.15.162:0/2254741542 >> 172.21.15.162:0/2254741542 conn(0x7f99000fd320 msgr2=0x7f99000ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.251+0000 7f9907963640 1 -- 172.21.15.162:0/2254741542 shutdown_connections 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.251+0000 7f9907963640 1 -- 172.21.15.162:0/2254741542 wait complete. 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9907963640 1 Processor -- start 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9907963640 1 -- start start 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9907963640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 0x7f99001973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9907963640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 0x7f990019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:30.254 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9907963640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f990010bd20 con 0x7f9900107660 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9907963640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f990010bba0 con 0x7f9900197900 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9904efe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 0x7f990019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9904efe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 0x7f990019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47920/0 (socket says 172.21.15.162:47920) 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9904efe640 1 -- 172.21.15.162:0/1063696546 learned_addr learned my addr 172.21.15.162:0/1063696546 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f99056ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 0x7f99001973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9904efe640 1 -- 172.21.15.162:0/1063696546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 msgr2=0x7f990019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9904efe640 1 -- 172.21.15.162:0/1063696546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 msgr2=0x7f990019bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9904efe640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 0x7f990019bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f9904efe640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 0x7f990019bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f99056ff640 1 -- 172.21.15.162:0/1063696546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 msgr2=0x7f990019bc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.252+0000 7f99056ff640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 0x7f990019bc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.255 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.253+0000 7f99056ff640 1 -- 172.21.15.162:0/1063696546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9900197f20 con 0x7f9900107660 2024-09-10T09:55:30.256 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.253+0000 7f99056ff640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 0x7f99001973c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98f4006010 tx=0x7f98f4032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.253+0000 7f98f27fc640 1 -- 172.21.15.162:0/1063696546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f98f4035070 con 0x7f9900107660 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.253+0000 7f98f27fc640 1 -- 172.21.15.162:0/1063696546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f4032e40 con 0x7f9900107660 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.253+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990019c1d0 con 0x7f9900107660 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.253+0000 7f98f27fc640 1 -- 172.21.15.162:0/1063696546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f98f403bb50 con 0x7f9900107660 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.253+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990019c710 con 0x7f9900107660 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.254+0000 7f98f27fc640 1 -- 172.21.15.162:0/1063696546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f98f4042030 con 0x7f9900107660 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.254+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9900102da0 con 0x7f9900107660 2024-09-10T09:55:30.257 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.254+0000 7f98f27fc640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98ec038e00 0x7f98ec03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:30.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.254+0000 7f98f27fc640 1 -- 172.21.15.162:0/1063696546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98f4040070 con 0x7f9900107660 2024-09-10T09:55:30.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.254+0000 7f9904efe640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98ec038e00 0x7f98ec03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:30.258 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.255+0000 7f9904efe640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98ec038e00 0x7f98ec03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009100 tx=0x7f98fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:30.260 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.259+0000 7f98f27fc640 1 -- 172.21.15.162:0/1063696546 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98f404a260 con 0x7f9900107660 2024-09-10T09:55:30.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.488+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f990019c9c0 con 0x7f9900107660 2024-09-10T09:55:30.490 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.489+0000 7f98f27fc640 1 -- 172.21.15.162:0/1063696546 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f98f403b460 con 0x7f9900107660 2024-09-10T09:55:30.490 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:55:30.490 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":2,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","modified":"2024-09-10T09:55:22.418058Z","created":"2024-09-10T09:53:50.319028Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:55:30.491 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 2 2024-09-10T09:55:30.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98ec038e00 msgr2=0x7f98ec03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98ec038e00 0x7f98ec03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009100 tx=0x7f98fc008040 comp rx=0 tx=0).stop 2024-09-10T09:55:30.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 msgr2=0x7f99001973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:30.492 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 0x7f99001973c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f98f4006010 tx=0x7f98f4032c50 comp rx=0 tx=0).stop 2024-09-10T09:55:30.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 shutdown_connections 2024-09-10T09:55:30.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98ec038e00 0x7f98ec03b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9900197900 0x7f990019bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 --2- 172.21.15.162:0/1063696546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9900107660 0x7f99001973c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:30.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 >> 172.21.15.162:0/1063696546 conn(0x7f99000fd320 msgr2=0x7f99000fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:30.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 shutdown_connections 2024-09-10T09:55:30.493 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:30.491+0000 7f9907963640 1 -- 172.21.15.162:0/1063696546 wait complete. 2024-09-10T09:55:30.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:30 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/1063696546' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:30.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:30 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/1063696546' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:31.182 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:31 smithi047 ceph-mon[35244]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T09:55:31.182 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:31 smithi047 bash[35225]: 267f885128f0d6198c2d76ccc2e64402aa5a38064b1f1e67150d9e2bb228c081 2024-09-10T09:55:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:31 smithi047 systemd[1]: Started Ceph mon.b for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T09:55:31.993 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:55:31.993 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:55:32.243 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.306+0000 7f750d125640 1 -- 172.21.15.162:0/628907878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 msgr2=0x7f74f40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.306+0000 7f750d125640 1 --2- 172.21.15.162:0/628907878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 0x7f74f40059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f75081016a0 tx=0x7f74f802c3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.306+0000 7f750d125640 1 -- 172.21.15.162:0/628907878 shutdown_connections 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.306+0000 7f750d125640 1 --2- 172.21.15.162:0/628907878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 0x7f74f40059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.306+0000 7f750d125640 1 -- 172.21.15.162:0/628907878 >> 172.21.15.162:0/628907878 conn(0x7f75080fc220 msgr2=0x7f75080fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.307+0000 7f750d125640 1 -- 172.21.15.162:0/628907878 shutdown_connections 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.307+0000 7f750d125640 1 -- 172.21.15.162:0/628907878 wait complete. 2024-09-10T09:55:36.309 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.308+0000 7f750d125640 1 Processor -- start 2024-09-10T09:55:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.308+0000 7f750d125640 1 -- start start 2024-09-10T09:55:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.308+0000 7f750d125640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 0x7f75081972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.308+0000 7f750d125640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 0x7f750819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.311 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.308+0000 7f750d125640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 0x7f750819c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.308+0000 7f750d125640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7508108670 con 0x7f74f4003530 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.308+0000 7f750d125640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f75081084f0 con 0x7f750819c0e0 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f750d125640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f75081087f0 con 0x7f7508197810 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f74fe59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 0x7f750819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f7506d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 0x7f75081972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f74fe59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 0x7f750819bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:47934/0 (socket says 172.21.15.162:47934) 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f74fe59c640 1 -- 172.21.15.162:0/3168668192 learned_addr learned my addr 172.21.15.162:0/3168668192 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f750759e640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 0x7f750819c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f74fe59c640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 msgr2=0x7f750819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f74fe59c640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 msgr2=0x7f750819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:55:36.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f74fe59c640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 0x7f750819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f74fe59c640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 0x7f750819bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f750759e640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 msgr2=0x7f750819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f750759e640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 msgr2=0x7f750819c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f750759e640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 0x7f750819c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f750759e640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 0x7f750819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f7506d9d640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 msgr2=0x7f750819bba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f7506d9d640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 0x7f750819bba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f7506d9d640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 msgr2=0x7f750819c590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f7506d9d640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 0x7f750819c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.309+0000 7f7506d9d640 1 -- 172.21.15.162:0/3168668192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75081a3e50 con 0x7f74f4003530 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.310+0000 7f7506d9d640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 0x7f75081972d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005f10 tx=0x7f74f80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.310+0000 7f7504d99640 1 -- 172.21.15.162:0/3168668192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8012450 con 0x7f74f4003530 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.310+0000 7f7504d99640 1 -- 172.21.15.162:0/3168668192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f8042070 con 0x7f74f4003530 2024-09-10T09:55:36.313 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.310+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75081a4020 con 0x7f74f4003530 2024-09-10T09:55:36.314 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.310+0000 7f7504d99640 1 -- 172.21.15.162:0/3168668192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f803d8f0 con 0x7f74f4003530 2024-09-10T09:55:36.314 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.310+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75081a4600 con 0x7f74f4003530 2024-09-10T09:55:36.314 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.311+0000 7f7504d99640 1 -- 172.21.15.162:0/3168668192 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f74f803da90 con 0x7f74f4003530 2024-09-10T09:55:36.314 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.311+0000 7f7504d99640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74e80394f0 0x7f74e803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.314 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.311+0000 7f7504d99640 1 -- 172.21.15.162:0/3168668192 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f74f806f670 con 0x7f74f4003530 2024-09-10T09:55:36.315 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.311+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75081097a0 con 0x7f74f4003530 2024-09-10T09:55:36.315 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.312+0000 7f74fe59c640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74e80394f0 0x7f74e803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.315 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.312+0000 7f74fe59c640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74e80394f0 0x7f74e803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009100 tx=0x7f74f0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.318 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.317+0000 7f7504d99640 1 -- 172.21.15.162:0/3168668192 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74f8045c20 con 0x7f74f4003530 2024-09-10T09:55:36.560 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.559+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f75081a4830 con 0x7f74f4003530 2024-09-10T09:55:36.561 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:36.561 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:55:36.561 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: mon.a calling monitor election 2024-09-10T09:55:36.561 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: mon.c calling monitor election 2024-09-10T09:55:36.561 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.561 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: monmap epoch 3 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:36.562 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: last_changed 2024-09-10T09:55:31.177760+0000 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: min_mon_release 19 (squid) 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: election_strategy: 1 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: fsmap 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: mgrmap e13: x(active, since 47s) 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: overall HEALTH_OK 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:36.563 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:36.564 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.560+0000 7f7504d99640 1 -- 172.21.15.162:0/3168668192 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f74f8034de0 con 0x7f74f4003530 2024-09-10T09:55:36.564 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:55:36.564 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":3,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","modified":"2024-09-10T09:55:31.177760Z","created":"2024-09-10T09:53:50.319028Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:55:36.565 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 3 2024-09-10T09:55:36.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74e80394f0 msgr2=0x7f74e803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74e80394f0 0x7f74e803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009100 tx=0x7f74f0008040 comp rx=0 tx=0).stop 2024-09-10T09:55:36.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 msgr2=0x7f75081972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 0x7f75081972d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005f10 tx=0x7f74f80048d0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.565 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 shutdown_connections 2024-09-10T09:55:36.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74e80394f0 0x7f74e803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f750819c0e0 0x7f750819c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7508197810 0x7f750819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 --2- 172.21.15.162:0/3168668192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74f4003530 0x7f75081972d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 >> 172.21.15.162:0/3168668192 conn(0x7f75080fc220 msgr2=0x7f7508105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 shutdown_connections 2024-09-10T09:55:36.566 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:36.563+0000 7f750d125640 1 -- 172.21.15.162:0/3168668192 wait complete. 2024-09-10T09:55:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:55:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: mon.a calling monitor election 2024-09-10T09:55:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: mon.c calling monitor election 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: monmap epoch 3 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:36.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: last_changed 2024-09-10T09:55:31.177760+0000 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: min_mon_release 19 (squid) 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: election_strategy: 1 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: fsmap 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: mgrmap e13: x(active, since 47s) 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: overall HEALTH_OK 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:36.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:36.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:37.139 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T09:55:37.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T09:55:37.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:37.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:37 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/3168668192' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:37.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:37.409 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:37.530 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:37.531 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:37 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/3168668192' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:37.531 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:37.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.842+0000 7ff74df0a640 1 -- 172.21.15.45:0/877024442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff748107580 msgr2=0x7ff748107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.842+0000 7ff74df0a640 1 --2- 172.21.15.45:0/877024442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff748107580 0x7ff748107960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff738009880 tx=0x7ff73802a5d0 comp rx=0 tx=0).stop 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.842+0000 7ff74df0a640 1 -- 172.21.15.45:0/877024442 shutdown_connections 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.842+0000 7ff74df0a640 1 --2- 172.21.15.45:0/877024442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff748107580 0x7ff748107960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.842+0000 7ff74df0a640 1 -- 172.21.15.45:0/877024442 >> 172.21.15.45:0/877024442 conn(0x7ff7480fb920 msgr2=0x7ff7480fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.842+0000 7ff74df0a640 1 -- 172.21.15.45:0/877024442 shutdown_connections 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.842+0000 7ff74df0a640 1 -- 172.21.15.45:0/877024442 wait complete. 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.843+0000 7ff74df0a640 1 Processor -- start 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.843+0000 7ff74df0a640 1 -- start start 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff74df0a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 0x7ff748100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff74df0a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7481011f0 0x7ff748101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff74df0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7481a3550 0x7ff7481a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:37.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff74df0a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff74810bc40 con 0x7ff7481a3550 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff74df0a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff74810bac0 con 0x7ff748107580 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff74df0a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff74810bdc0 con 0x7ff7481011f0 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7481a3550 0x7ff7481a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7481a3550 0x7ff7481a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52840/0 (socket says 172.21.15.45:52840) 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 0x7ff748100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 -- 172.21.15.45:0/2775198760 learned_addr learned my addr 172.21.15.45:0/2775198760 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff746ffd640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7481011f0 0x7ff748101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff7477fe640 1 -- 172.21.15.45:0/2775198760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 msgr2=0x7ff748100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff7477fe640 1 -- 172.21.15.45:0/2775198760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 msgr2=0x7ff748100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff7477fe640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 0x7ff748100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff7477fe640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 0x7ff748100cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 -- 172.21.15.45:0/2775198760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7481011f0 msgr2=0x7ff748101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7481011f0 0x7ff748101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 -- 172.21.15.45:0/2775198760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 msgr2=0x7ff748100cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:55:37.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 0x7ff748100cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 -- 172.21.15.45:0/2775198760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7481a6010 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.844+0000 7ff747fff640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7481a3550 0x7ff7481a5940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff74000ba50 tx=0x7ff74000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.845+0000 7ff744ff9640 1 -- 172.21.15.45:0/2775198760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740019070 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.845+0000 7ff744ff9640 1 -- 172.21.15.45:0/2775198760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff74000cbe0 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.845+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7481a6300 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.845+0000 7ff744ff9640 1 -- 172.21.15.45:0/2775198760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400076a0 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.845+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7481a6840 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.846+0000 7ff744ff9640 1 -- 172.21.15.45:0/2775198760 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff74001c070 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.846+0000 7ff744ff9640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff7300394f0 0x7ff73003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.846+0000 7ff744ff9640 1 -- 172.21.15.45:0/2775198760 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff74004ca90 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.846+0000 7ff7477fe640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff7300394f0 0x7ff73003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.846+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff748102dd0 con 0x7ff7481a3550 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.846+0000 7ff7477fe640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff7300394f0 0x7ff73003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff738005ab0 tx=0x7ff738005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:37.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:37.851+0000 7ff744ff9640 1 -- 172.21.15.45:0/2775198760 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff74000e0a0 con 0x7ff7481a3550 2024-09-10T09:55:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.033+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff7481a6b00 con 0x7ff7481a3550 2024-09-10T09:55:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.033+0000 7ff744ff9640 1 -- 172.21.15.45:0/2775198760 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7ff7400112a0 con 0x7ff7481a3550 2024-09-10T09:55:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:38.035 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-09-10T09:55:38.035 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:38.035 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-10T09:55:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff7300394f0 msgr2=0x7ff73003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff7300394f0 0x7ff73003b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff738005ab0 tx=0x7ff738005a20 comp rx=0 tx=0).stop 2024-09-10T09:55:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7481a3550 msgr2=0x7ff7481a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:38.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7481a3550 0x7ff7481a5940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff74000ba50 tx=0x7ff74000bf10 comp rx=0 tx=0).stop 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 shutdown_connections 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff7300394f0 0x7ff73003b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7481a3550 0x7ff7481a5940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff7481011f0 0x7ff748101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 --2- 172.21.15.45:0/2775198760 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff748107580 0x7ff748100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.036+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 >> 172.21.15.45:0/2775198760 conn(0x7ff7480fb920 msgr2=0x7ff7480fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.037+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 shutdown_connections 2024-09-10T09:55:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:38.037+0000 7ff74df0a640 1 -- 172.21.15.45:0/2775198760 wait complete. 2024-09-10T09:55:38.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:38.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:55:38.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.a calling monitor election 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.c calling monitor election 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: monmap epoch 3 2024-09-10T09:55:38.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: last_changed 2024-09-10T09:55:31.177760+0000 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: min_mon_release 19 (squid) 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: election_strategy: 1 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: fsmap 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mgrmap e13: x(active, since 47s) 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: overall HEALTH_OK 2024-09-10T09:55:38.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:38.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/3168668192' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:55:38.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:38.731 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T09:55:38.731 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:55:38.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:55:38.759 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:55:38.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:38.828 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:55:38.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:55:38.858 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:55:38.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.b calling monitor election 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.b calling monitor election 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.c calling monitor election 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.a calling monitor election 2024-09-10T09:55:38.921 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: monmap epoch 3 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: last_changed 2024-09-10T09:55:31.177760+0000 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: min_mon_release 19 (squid) 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: election_strategy: 1 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: fsmap 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: mgrmap e13: x(active, since 49s) 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: overall HEALTH_OK 2024-09-10T09:55:38.922 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:38.923 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:38.924 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:38.924 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:55:38.925 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:55:38.954 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:55:38.954 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:55:39.019 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-09-10T09:55:39.019 INFO:tasks.cephadm:Adding mgr.y on smithi047 2024-09-10T09:55:39.019 INFO:tasks.cephadm:Adding mgr.z on smithi162 2024-09-10T09:55:39.019 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi045=x;smithi047=y;smithi162=z' 2024-09-10T09:55:39.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: mon.b calling monitor election 2024-09-10T09:55:39.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-10T09:55:39.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: mon.b calling monitor election 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: mon.c calling monitor election 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: mon.a calling monitor election 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: monmap epoch 3 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: last_changed 2024-09-10T09:55:31.177760+0000 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: min_mon_release 19 (squid) 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: election_strategy: 1 2024-09-10T09:55:39.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: fsmap 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: mgrmap e13: x(active, since 49s) 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: overall HEALTH_OK 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:39.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.293 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: mon.b calling monitor election 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: mon.b calling monitor election 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: mon.c calling monitor election 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: mon.a calling monitor election 2024-09-10T09:55:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: monmap epoch 3 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: last_changed 2024-09-10T09:55:31.177760+0000 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: created 2024-09-10T09:53:50.319028+0000 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: min_mon_release 19 (squid) 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: election_strategy: 1 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: 1: [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon.c 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: 2: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.b 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: fsmap 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: mgrmap e13: x(active, since 49s) 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: overall HEALTH_OK 2024-09-10T09:55:39.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:39.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:39.403 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:39.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.692+0000 7f4c90a09640 1 -- 172.21.15.162:0/1938418498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c102180 msgr2=0x7f4c8c104570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:39.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.692+0000 7f4c90a09640 1 --2- 172.21.15.162:0/1938418498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c102180 0x7f4c8c104570 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009880 tx=0x7f4c8002a5d0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.693+0000 7f4c90a09640 1 -- 172.21.15.162:0/1938418498 shutdown_connections 2024-09-10T09:55:39.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.693+0000 7f4c90a09640 1 --2- 172.21.15.162:0/1938418498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c102180 0x7f4c8c104570 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.697 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.693+0000 7f4c90a09640 1 -- 172.21.15.162:0/1938418498 >> 172.21.15.162:0/1938418498 conn(0x7f4c8c0fbda0 msgr2=0x7f4c8c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.693+0000 7f4c90a09640 1 -- 172.21.15.162:0/1938418498 shutdown_connections 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.693+0000 7f4c90a09640 1 -- 172.21.15.162:0/1938418498 wait complete. 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.693+0000 7f4c90a09640 1 Processor -- start 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.693+0000 7f4c90a09640 1 -- start start 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c90a09640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8c102180 0x7f4c8c101ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c90a09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c100130 0x7f4c8c1005b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c90a09640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8c19ab70 0x7f4c8c100af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c90a09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c107740 con 0x7f4c8c100130 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c90a09640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1075c0 con 0x7f4c8c19ab70 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c90a09640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1078c0 con 0x7f4c8c102180 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c8a59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8c102180 0x7f4c8c101ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c8a59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8c102180 0x7f4c8c101ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34710/0 (socket says 172.21.15.162:34710) 2024-09-10T09:55:39.698 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c8a59c640 1 -- 172.21.15.162:0/2473102000 learned_addr learned my addr 172.21.15.162:0/2473102000 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c89d9b640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c100130 0x7f4c8c1005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c8ad9d640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8c19ab70 0x7f4c8c100af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c89d9b640 1 -- 172.21.15.162:0/2473102000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8c102180 msgr2=0x7f4c8c101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c89d9b640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8c102180 0x7f4c8c101ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c89d9b640 1 -- 172.21.15.162:0/2473102000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8c19ab70 msgr2=0x7f4c8c100af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c89d9b640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8c19ab70 0x7f4c8c100af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c89d9b640 1 -- 172.21.15.162:0/2473102000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c1a5940 con 0x7f4c8c100130 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.694+0000 7f4c8a59c640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8c102180 0x7f4c8c101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.695+0000 7f4c89d9b640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c100130 0x7f4c8c1005b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00b450 tx=0x7f4c7c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.695+0000 7f4c7b7fe640 1 -- 172.21.15.162:0/2473102000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c004230 con 0x7f4c8c100130 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.695+0000 7f4c7b7fe640 1 -- 172.21.15.162:0/2473102000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7c0043d0 con 0x7f4c8c100130 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.695+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c1a5c30 con 0x7f4c8c100130 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.695+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a61c0 con 0x7f4c8c100130 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.695+0000 7f4c7b7fe640 1 -- 172.21.15.162:0/2473102000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c003cb0 con 0x7f4c8c100130 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.696+0000 7f4c7b7fe640 1 -- 172.21.15.162:0/2473102000 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4c7c00f050 con 0x7f4c8c100130 2024-09-10T09:55:39.699 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.696+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c190500 con 0x7f4c8c100130 2024-09-10T09:55:39.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.696+0000 7f4c7b7fe640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4c74039520 0x7f4c7403b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:39.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.696+0000 7f4c7b7fe640 1 -- 172.21.15.162:0/2473102000 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4c7c04bc20 con 0x7f4c8c100130 2024-09-10T09:55:39.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.696+0000 7f4c8a59c640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4c74039520 0x7f4c7403b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:39.700 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.697+0000 7f4c8a59c640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4c74039520 0x7f4c7403b9e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c8002abc0 tx=0x7f4c800058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:39.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.701+0000 7f4c7b7fe640 1 -- 172.21.15.162:0/2473102000 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c7c00d040 con 0x7f4c8c100130 2024-09-10T09:55:39.877 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.876+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi047=y;smithi162=z", "target": ["mon-mgr", ""]}) -- 0x7f4c8c0626c0 con 0x7f4c74039520 2024-09-10T09:55:39.883 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.882+0000 7f4c7b7fe640 1 -- 172.21.15.162:0/2473102000 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4c8c0626c0 con 0x7f4c74039520 2024-09-10T09:55:39.883 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mgr update... 2024-09-10T09:55:39.885 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4c74039520 msgr2=0x7f4c7403b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:39.885 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4c74039520 0x7f4c7403b9e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c8002abc0 tx=0x7f4c800058d0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c100130 msgr2=0x7f4c8c1005b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c100130 0x7f4c8c1005b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00b450 tx=0x7f4c7c00b910 comp rx=0 tx=0).stop 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 shutdown_connections 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4c74039520 0x7f4c7403b9e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4c8c19ab70 0x7f4c8c100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c8c100130 0x7f4c8c1005b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 --2- 172.21.15.162:0/2473102000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4c8c102180 0x7f4c8c101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 >> 172.21.15.162:0/2473102000 conn(0x7f4c8c0fbda0 msgr2=0x7f4c8c0fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 shutdown_connections 2024-09-10T09:55:39.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:55:39.884+0000 7f4c90a09640 1 -- 172.21.15.162:0/2473102000 wait complete. 2024-09-10T09:55:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:39 smithi162 ceph-mon[34963]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:40.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:39 smithi162 ceph-mon[34963]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:55:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:39 smithi162 ceph-mon[34963]: Reconfiguring daemon mon.a on smithi045 2024-09-10T09:55:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:39 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:39 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:40.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:39 smithi047 ceph-mon[35244]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:40.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:39 smithi047 ceph-mon[35244]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:55:40.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:39 smithi047 ceph-mon[35244]: Reconfiguring daemon mon.a on smithi045 2024-09-10T09:55:40.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:39 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:40.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:39 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:39 smithi045 ceph-mon[31114]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:39 smithi045 ceph-mon[31114]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:55:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:39 smithi045 ceph-mon[31114]: Reconfiguring daemon mon.a on smithi045 2024-09-10T09:55:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:39 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:55:40.390 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:39 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:40.466 DEBUG:teuthology.orchestra.run.smithi047:mgr.y> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T09:55:40.469 DEBUG:teuthology.orchestra.run.smithi162:mgr.z> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T09:55:40.471 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T09:55:40.471 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T09:55:40.472 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:55:40.488 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:55:40.488 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-09-10T09:55:40.544 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:55:40.544 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:40.544 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T09:55:40.544 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:40.544 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:40.545 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-10 09:54:59.670819101 +0000 2024-09-10T09:55:40.545 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-10 09:50:17.495424170 +0000 2024-09-10T09:55:40.545 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-10 09:50:17.495424170 +0000 2024-09-10T09:55:40.545 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-10 09:50:17.495424170 +0000 2024-09-10T09:55:40.545 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:55:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-10T09:55:40.611 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-10T09:55:40.612 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000126765 s, 4.0 MB/s 2024-09-10T09:55:40.613 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:55:40.669 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-10 09:54:59.700818597 +0000 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-10 09:50:17.810419148 +0000 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-10 09:50:17.810419148 +0000 2024-09-10T09:55:40.726 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-10 09:50:17.810419148 +0000 2024-09-10T09:55:40.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:55:40.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-10T09:55:40.790 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-10T09:55:40.790 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.2053e-05 s, 5.6 MB/s 2024-09-10T09:55:40.791 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:55:40.848 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-09-10T09:55:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:55:40.905 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:40.905 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-10T09:55:40.905 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:40.905 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:40.905 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-10 09:54:59.730818092 +0000 2024-09-10T09:55:40.906 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-10 09:50:18.130414046 +0000 2024-09-10T09:55:40.906 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-10 09:50:18.130414046 +0000 2024-09-10T09:55:40.906 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-10 09:50:18.130414046 +0000 2024-09-10T09:55:40.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:55:40.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-10T09:55:40.970 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-10T09:55:40.970 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000104605 s, 4.9 MB/s 2024-09-10T09:55:40.971 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:55:41.029 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-09-10T09:55:41.084 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-10 09:54:59.760817587 +0000 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-10 09:50:18.449408960 +0000 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-10 09:50:18.449408960 +0000 2024-09-10T09:55:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-10 09:50:18.449408960 +0000 2024-09-10T09:55:41.085 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:55:41.149 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-09-10T09:55:41.149 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-09-10T09:55:41.149 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000108498 s, 4.7 MB/s 2024-09-10T09:55:41.150 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:55:41.207 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-09-10T09:55:41.208 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:55:41.227 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:55:41.227 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 09:55:30.628631069 +0000 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 09:50:31.774852061 +0000 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 09:50:31.774852061 +0000 2024-09-10T09:55:41.286 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 09:50:31.774852061 +0000 2024-09-10T09:55:41.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:55:41.351 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi047=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:41.351 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: Saving service mgr spec with placement smithi045=x;smithi047=y;smithi162=z;count:3 2024-09-10T09:55:41.351 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:41.351 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:41.351 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:55:41.351 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:41.351 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:41.352 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:41.352 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:41 smithi047 ceph-mon[35244]: Reconfiguring daemon mgr.x on smithi045 2024-09-10T09:55:41.354 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T09:55:41.354 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T09:55:41.354 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000116525 s, 4.4 MB/s 2024-09-10T09:55:41.355 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:55:41.371 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi047=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:41.371 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: Saving service mgr spec with placement smithi045=x;smithi047=y;smithi162=z;count:3 2024-09-10T09:55:41.371 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:41.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:41.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:55:41.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:41.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:41.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:41.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:41 smithi045 ceph-mon[31114]: Reconfiguring daemon mgr.x on smithi045 2024-09-10T09:55:41.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi047=y;smithi162=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:41.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: Saving service mgr spec with placement smithi045=x;smithi047=y;smithi162=z;count:3 2024-09-10T09:55:41.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:55:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:41 smithi162 ceph-mon[34963]: Reconfiguring daemon mgr.x on smithi045 2024-09-10T09:55:41.414 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 09:55:30.629631047 +0000 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 09:50:32.093845246 +0000 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 09:50:32.093845246 +0000 2024-09-10T09:55:41.471 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 09:50:32.093845246 +0000 2024-09-10T09:55:41.471 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:55:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T09:55:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T09:55:41.535 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.2964e-05 s, 5.5 MB/s 2024-09-10T09:55:41.536 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:55:41.592 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-09-10T09:55:41.647 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:55:41.647 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:41.647 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T09:55:41.647 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:41.647 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:41.647 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 09:55:30.629631047 +0000 2024-09-10T09:55:41.647 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 09:50:32.416838346 +0000 2024-09-10T09:55:41.648 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 09:50:32.416838346 +0000 2024-09-10T09:55:41.648 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 09:50:32.416838346 +0000 2024-09-10T09:55:41.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:55:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T09:55:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T09:55:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 9.369e-05 s, 5.5 MB/s 2024-09-10T09:55:41.713 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:55:41.769 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-09-10 09:55:30.629631047 +0000 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-09-10 09:50:32.748831251 +0000 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-09-10 09:50:32.748831251 +0000 2024-09-10T09:55:41.826 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-09-10 09:50:32.748831251 +0000 2024-09-10T09:55:41.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:55:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-09-10T09:55:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-09-10T09:55:41.889 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000103338 s, 5.0 MB/s 2024-09-10T09:55:41.890 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:55:41.947 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T09:55:41.948 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:55:41.967 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:55:41.967 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 09:55:21.880851209 +0000 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 09:50:26.718967805 +0000 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 09:50:26.718967805 +0000 2024-09-10T09:55:42.027 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 09:50:26.718967805 +0000 2024-09-10T09:55:42.028 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:55:42.096 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T09:55:42.096 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T09:55:42.096 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.00033477 s, 1.5 MB/s 2024-09-10T09:55:42.097 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:55:42.158 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 09:55:21.880851209 +0000 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 09:50:27.047961254 +0000 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 09:50:27.047961254 +0000 2024-09-10T09:55:42.216 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 09:50:27.047961254 +0000 2024-09-10T09:55:42.217 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:55:42.284 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T09:55:42.284 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T09:55:42.284 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000238677 s, 2.1 MB/s 2024-09-10T09:55:42.286 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:55:42.346 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-09-10T09:55:42.404 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 09:55:21.880851209 +0000 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 09:50:27.374954743 +0000 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 09:50:27.374954743 +0000 2024-09-10T09:55:42.405 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 09:50:27.374954743 +0000 2024-09-10T09:55:42.405 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:55:42.473 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T09:55:42.473 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T09:55:42.473 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000422929 s, 1.2 MB/s 2024-09-10T09:55:42.475 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:55:42.533 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-09-10T09:55:42.591 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:55:42.591 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:55:42.592 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T09:55:42.592 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:55:42.592 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:55:42.592 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 09:55:21.881851189 +0000 2024-09-10T09:55:42.592 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 09:50:27.709948073 +0000 2024-09-10T09:55:42.592 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 09:50:27.709948073 +0000 2024-09-10T09:55:42.592 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 09:50:27.709948073 +0000 2024-09-10T09:55:42.592 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:55:42.658 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T09:55:42.658 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T09:55:42.659 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000228211 s, 2.2 MB/s 2024-09-10T09:55:42.660 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:55:42.717 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-09-10T09:55:42.720 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:42.749 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:42 smithi045 ceph-mon[31114]: Reconfiguring daemon mon.b on smithi047 2024-09-10T09:55:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:55:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:42 smithi162 ceph-mon[34963]: Reconfiguring daemon mon.b on smithi047 2024-09-10T09:55:42.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:42.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:42.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:42.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:55:42.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:42.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:42.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:42.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:42 smithi047 ceph-mon[35244]: Reconfiguring daemon mon.b on smithi047 2024-09-10T09:55:42.981 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:43.988 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:43 smithi162 ceph-mon[34963]: Reconfiguring daemon mon.c on smithi162 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:44.118 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:43 smithi045 ceph-mon[31114]: Reconfiguring daemon mon.c on smithi162 2024-09-10T09:55:44.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:44.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:44.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:44.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:55:44.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:55:44.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:55:44.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:44.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:43 smithi047 ceph-mon[35244]: Reconfiguring daemon mon.c on smithi162 2024-09-10T09:55:45.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:45 smithi045 ceph-mon[31114]: Deploying daemon mgr.z on smithi162 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:45.394 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:45 smithi162 ceph-mon[34963]: Deploying daemon mgr.z on smithi162 2024-09-10T09:55:45.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:45 smithi047 ceph-mon[35244]: Deploying daemon mgr.z on smithi162 2024-09-10T09:55:45.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:55:45.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-09-10T09:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:46.021 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:45 smithi162 systemd[1]: Starting Ceph mgr.z for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T09:55:46.220 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:46 smithi045 ceph-mon[31114]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:46.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 -- 172.21.15.45:0/1068330896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 msgr2=0x7f85d41002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 --2- 172.21.15.45:0/1068330896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d41002d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009930 tx=0x7f85c402a420 comp rx=0 tx=0).stop 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 -- 172.21.15.45:0/1068330896 shutdown_connections 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 --2- 172.21.15.45:0/1068330896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85d4103870 0x7f85d410a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 --2- 172.21.15.45:0/1068330896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4102eb0 0x7f85d4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 --2- 172.21.15.45:0/1068330896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d41002d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 -- 172.21.15.45:0/1068330896 >> 172.21.15.45:0/1068330896 conn(0x7f85d40fba40 msgr2=0x7f85d40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 -- 172.21.15.45:0/1068330896 shutdown_connections 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.305+0000 7f85d8e97640 1 -- 172.21.15.45:0/1068330896 wait complete. 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 Processor -- start 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 -- start start 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:46.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85d4102eb0 0x7f85d419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4103870 0x7f85d41a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f85d410fb20 con 0x7f85d40ffed0 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85d410f9a0 con 0x7f85d4102eb0 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d8e97640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f85d410fca0 con 0x7f85d4103870 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d1d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85d4102eb0 0x7f85d419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53536/0 (socket says 172.21.15.45:53536) 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d259c640 1 -- 172.21.15.45:0/798386950 learned_addr learned my addr 172.21.15.45:0/798386950 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.306+0000 7f85d2d9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4103870 0x7f85d41a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d259c640 1 -- 172.21.15.45:0/798386950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4103870 msgr2=0x7f85d41a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d259c640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4103870 0x7f85d41a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d259c640 1 -- 172.21.15.45:0/798386950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85d4102eb0 msgr2=0x7f85d419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:46.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d259c640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85d4102eb0 0x7f85d419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d259c640 1 -- 172.21.15.45:0/798386950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d41a1d90 con 0x7f85d40ffed0 2024-09-10T09:55:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d259c640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d419a160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85c4031610 tx=0x7f85c4032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c403f030 con 0x7f85d40ffed0 2024-09-10T09:55:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4032e50 con 0x7f85d40ffed0 2024-09-10T09:55:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d41a2020 con 0x7f85d40ffed0 2024-09-10T09:55:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4035070 con 0x7f85d40ffed0 2024-09-10T09:55:46.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.307+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d41a2500 con 0x7f85d40ffed0 2024-09-10T09:55:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.308+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f85c4042030 con 0x7f85d40ffed0 2024-09-10T09:55:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.308+0000 7f85c37fe640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f85bc0392c0 0x7f85bc03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.308+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d4109100 con 0x7f85d40ffed0 2024-09-10T09:55:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.308+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f85c406c4d0 con 0x7f85d40ffed0 2024-09-10T09:55:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.308+0000 7f85d1d9b640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f85bc0392c0 0x7f85bc03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:46.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.309+0000 7f85d1d9b640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f85bc0392c0 0x7f85bc03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f85d419b680 tx=0x7f85c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:46.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.313+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85c4039e20 con 0x7f85d40ffed0 2024-09-10T09:55:46.315 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-mon[34963]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:46.315 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 podman[36757]: 2024-09-10 09:55:46.020337834 +0000 UTC m=+0.095410909 container create 94a2f2b5237c25391e5e38ce914f9ac63863ce384a5b51e5add69700d68db61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T09:55:46.315 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 podman[36757]: 2024-09-10 09:55:45.940374787 +0000 UTC m=+0.015447864 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:55:46.315 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 podman[36757]: 2024-09-10 09:55:46.23097161 +0000 UTC m=+0.306044687 container init 94a2f2b5237c25391e5e38ce914f9ac63863ce384a5b51e5add69700d68db61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T09:55:46.315 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 podman[36757]: 2024-09-10 09:55:46.233679019 +0000 UTC m=+0.308752096 container start 94a2f2b5237c25391e5e38ce914f9ac63863ce384a5b51e5add69700d68db61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T09:55:46.315 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-mgr[36777]: -- 172.21.15.162:0/2838182690 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fcaaa50e0 con 0x564fcaa82800 2024-09-10T09:55:46.315 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 bash[36757]: 94a2f2b5237c25391e5e38ce914f9ac63863ce384a5b51e5add69700d68db61a 2024-09-10T09:55:46.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:46 smithi047 ceph-mon[35244]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:46.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:46.501+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f85d4062630 con 0x7f85bc0392c0 2024-09-10T09:55:46.567 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 systemd[1]: Started Ceph mgr.z for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T09:55:46.567 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:46.357+0000 7f7ba4301040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:55:46.567 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:46.417+0000 7f7ba4301040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:55:46.567 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:46.566+0000 7f7ba4301040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:55:46.899 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:46.631+0000 7f7ba4301040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:55:46.899 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:46.778+0000 7f7ba4301040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:55:47.235 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:46.918+0000 7f7ba4301040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:55:47.235 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:47.234+0000 7f7ba4301040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:55:47.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: Deploying daemon mgr.y on smithi047 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:55:47.561 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:47.649 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:47.295+0000 7f7ba4301040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:55:47.649 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:47.479+0000 7f7ba4301040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: Deploying daemon mgr.y on smithi047 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:55:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:47.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: Deploying daemon mgr.y on smithi047 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:55:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:48.324 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 systemd[1]: Starting Ceph mgr.y for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T09:55:48.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:48 smithi045 ceph-mon[31114]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:48.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:48 smithi045 ceph-mon[31114]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:48.576 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:48 smithi047 ceph-mon[35244]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:48.576 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:48 smithi047 ceph-mon[35244]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:48.577 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 podman[36591]: 2024-09-10 09:55:48.226070796 +0000 UTC m=+0.016139222 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:55:48.577 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 podman[36591]: 2024-09-10 09:55:48.332491634 +0000 UTC m=+0.122560051 container create dd075593474f4d01697da219a355f1762a3683b9e4b8a1b44e805b00388cc3ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T09:55:48.577 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 podman[36591]: 2024-09-10 09:55:48.509906255 +0000 UTC m=+0.299974672 container init dd075593474f4d01697da219a355f1762a3683b9e4b8a1b44e805b00388cc3ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T09:55:48.577 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 podman[36591]: 2024-09-10 09:55:48.512492919 +0000 UTC m=+0.302561333 container start dd075593474f4d01697da219a355f1762a3683b9e4b8a1b44e805b00388cc3ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T09:55:48.577 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 ceph-mgr[36610]: -- 172.21.15.47:0/245452690 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4971070e0 con 0x55a4970e5400 2024-09-10T09:55:48.577 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 bash[36591]: dd075593474f4d01697da219a355f1762a3683b9e4b8a1b44e805b00388cc3ea 2024-09-10T09:55:48.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:48 smithi162 ceph-mon[34963]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:55:48.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:48 smithi162 ceph-mon[34963]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:48.650 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:48.363+0000 7f7ba4301040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:55:48.650 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:48.431+0000 7f7ba4301040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:55:48.860 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 systemd[1]: Started Ceph mgr.y for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T09:55:48.860 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:48.639+0000 7f2faa07e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:55:48.860 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:48.701+0000 7f2faa07e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:55:48.860 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:48.858+0000 7f2faa07e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:55:49.160 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:48.929+0000 7f2faa07e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:55:49.160 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:49.092+0000 7f2faa07e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:55:49.477 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.207+0000 7f7ba4301040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:55:49.477 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.324+0000 7f7ba4301040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:55:49.477 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.475+0000 7f7ba4301040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:55:49.579 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:49.256+0000 7f2faa07e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.538+0000 7f7ba4301040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.692+0000 7f7ba4301040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.760+0000 7f7ba4301040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:49.762 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/143977778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edb5ca81-e36d-4acd-9f67-35e5fb644772"}]: dispatch 2024-09-10T09:55:49.763 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/143977778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edb5ca81-e36d-4acd-9f67-35e5fb644772"}]': finished 2024-09-10T09:55:49.763 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:55:49.763 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:55:49.763 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1903994000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:55:49.870 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:49.590+0000 7f2faa07e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:55:49.870 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:49.653+0000 7f2faa07e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:55:49.870 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:49.869+0000 7f2faa07e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:55:49.870 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:49.870 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.870 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/143977778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edb5ca81-e36d-4acd-9f67-35e5fb644772"}]: dispatch 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/143977778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edb5ca81-e36d-4acd-9f67-35e5fb644772"}]': finished 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:55:49.871 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:49 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1903994000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:55:49.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:49.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:49.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:49.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/143977778' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edb5ca81-e36d-4acd-9f67-35e5fb644772"}]: dispatch 2024-09-10T09:55:49.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/143977778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edb5ca81-e36d-4acd-9f67-35e5fb644772"}]': finished 2024-09-10T09:55:49.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:55:49.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:55:49.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:49 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1903994000' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:55:50.062 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.821+0000 7f7ba4301040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:55:50.063 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:49.994+0000 7f7ba4301040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:55:50.063 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:50.061+0000 7f7ba4301040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:55:50.339 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:50.132+0000 7f7ba4301040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:55:50.339 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:50.218+0000 7f7ba4301040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:55:50.339 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:50.278+0000 7f7ba4301040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:55:50.339 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:50.338+0000 7f7ba4301040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:55:51.128 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.128 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.128 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.128 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.128 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:51.128 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:51.128 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.129 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:51.129 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:51.129 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:51.129 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:51.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.159 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:50.789+0000 7f2faa07e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:55:51.159 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:50.859+0000 7f2faa07e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:51.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:51.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:51.399 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 09:55:51 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T09:55:51.127+0000 7f7ba4301040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:55:51.875 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:51.606+0000 7f2faa07e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:55:51.875 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:51.726+0000 7f2faa07e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:55:52.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:52.124+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f85c4005020 con 0x7f85d40ffed0 2024-09-10T09:55:52.159 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:51.874+0000 7f2faa07e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:51.937+0000 7f2faa07e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.092+0000 7f2faa07e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-mon[35244]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-mon[35244]: Standby manager daemon z started 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:55:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:55:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:52 smithi162 ceph-mon[34963]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:52.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:52 smithi162 ceph-mon[34963]: Standby manager daemon z started 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:52 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:52 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:52 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:52 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:52 smithi045 ceph-mon[31114]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:52 smithi045 ceph-mon[31114]: Standby manager daemon z started 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:52 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:52 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:55:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:52 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:55:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:52 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/890358463' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:55:52.458 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.163+0000 7f2faa07e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:55:52.458 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.226+0000 7f2faa07e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:55:52.458 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.390+0000 7f2faa07e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:55:52.458 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.457+0000 7f2faa07e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:55:52.734 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.526+0000 7f2faa07e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:55:52.734 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.611+0000 7f2faa07e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:55:52.734 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.673+0000 7f2faa07e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:55:52.734 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:52.733+0000 7f2faa07e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:55:53.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:53 smithi162 ceph-mon[34963]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T09:55:53.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:55:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:53 smithi045 ceph-mon[31114]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T09:55:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:55:53.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:53 smithi047 ceph-mon[35244]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-10T09:55:53.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:55:53.909 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 09:55:53 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T09:55:53.556+0000 7f2faa07e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:55:54.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:55:54.132+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f85c403b4f0 con 0x7f85d40ffed0 2024-09-10T09:55:54.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:54 smithi162 ceph-mon[34963]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:54 smithi162 ceph-mon[34963]: Standby manager daemon y started 2024-09-10T09:55:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:54 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:55:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:54 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:55:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:54 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:55:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:54 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:55:54.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:54 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:54 smithi045 ceph-mon[31114]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:54.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:54 smithi045 ceph-mon[31114]: Standby manager daemon y started 2024-09-10T09:55:54.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:54 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:55:54.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:54 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:55:54.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:54 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:55:54.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:54 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:55:54.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:54 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:54.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:54 smithi047 ceph-mon[35244]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:54.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:54 smithi047 ceph-mon[35244]: Standby manager daemon y started 2024-09-10T09:55:54.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:54 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:55:54.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:54 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:55:54.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:54 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:55:54.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:54 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/1309489322' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:55:54.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:54 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:55:55.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:55 smithi162 ceph-mon[34963]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-10T09:55:55.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:55:55.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:55 smithi045 ceph-mon[31114]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-10T09:55:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:55:55.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:55 smithi047 ceph-mon[35244]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-10T09:55:55.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:55:56.389 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:56 smithi045 ceph-mon[31114]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:56.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:56 smithi162 ceph-mon[34963]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:56.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:56 smithi047 ceph-mon[35244]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:58.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:58 smithi162 ceph-mon[34963]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:58.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:55:58.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:58.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:55:58 smithi162 ceph-mon[34963]: Deploying daemon osd.0 on smithi045 2024-09-10T09:55:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:58 smithi045 ceph-mon[31114]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:55:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:58.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:55:58 smithi045 ceph-mon[31114]: Deploying daemon osd.0 on smithi045 2024-09-10T09:55:58.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:58 smithi047 ceph-mon[35244]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:55:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:55:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:55:58 smithi047 ceph-mon[35244]: Deploying daemon osd.0 on smithi045 2024-09-10T09:56:00.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:00 smithi162 ceph-mon[34963]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:00.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:00 smithi045 ceph-mon[31114]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:00.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:00 smithi047 ceph-mon[35244]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:02.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:02 smithi047 ceph-mon[35244]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:02.559 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:02 smithi162 ceph-mon[34963]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:02.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:02 smithi045 ceph-mon[31114]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:04.405 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:04 smithi045 ceph-mon[31114]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:04.405 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:04.406 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:04.406 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:04.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:04 smithi047 ceph-mon[35244]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:04.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:04.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:04.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:04 smithi162 ceph-mon[34963]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:06.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:06 smithi047 ceph-mon[35244]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:06.582 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:06 smithi045 ceph-mon[31114]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:06.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:06 smithi162 ceph-mon[34963]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:07.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.258+0000 7f85c37fe640 1 -- 172.21.15.45:0/798386950 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f85d4062630 con 0x7f85bc0392c0 2024-09-10T09:56:07.260 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f85bc0392c0 msgr2=0x7f85bc03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f85bc0392c0 0x7f85bc03b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f85d419b680 tx=0x7f85c800b410 comp rx=0 tx=0).stop 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 msgr2=0x7f85d419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d419a160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85c4031610 tx=0x7f85c4032c60 comp rx=0 tx=0).stop 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 shutdown_connections 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f85bc0392c0 0x7f85bc03b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85d4103870 0x7f85d41a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:07.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85d4102eb0 0x7f85d419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 --2- 172.21.15.45:0/798386950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85d40ffed0 0x7f85d419a160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 >> 172.21.15.45:0/798386950 conn(0x7f85d40fba40 msgr2=0x7f85d41062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 shutdown_connections 2024-09-10T09:56:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:56:07.262+0000 7f85d8e97640 1 -- 172.21.15.45:0/798386950 wait complete. 2024-09-10T09:56:07.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:07.564 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:07.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:07.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:07.867 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T09:56:07.869 INFO:tasks.cephadm:Deploying osd.1 on smithi047 with /dev/vg_nvme/lv_4... 2024-09-10T09:56:07.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:56:08.115 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:56:08.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:08 smithi045 ceph-mon[31114]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:08.532 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:08 smithi047 ceph-mon[35244]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:08.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:08 smithi162 ceph-mon[34963]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:10.151 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:56:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0[44916]: 2024-09-10T09:56:09.847+0000 7f1c9e86d640 -1 osd.0 0 log_to_monitors true 2024-09-10T09:56:10.396 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:56:10.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-09-10T09:56:10.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:10 smithi045 ceph-mon[31114]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:10.510 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:10 smithi045 ceph-mon[31114]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:56:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:10 smithi162 ceph-mon[34963]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:10 smithi162 ceph-mon[34963]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:56:10.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:10 smithi047 ceph-mon[35244]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:10.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:10 smithi047 ceph-mon[35244]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:56:10.677 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:56:11.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 -- 172.21.15.47:0/1884409331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c100ad0 msgr2=0x7fae9c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 --2- 172.21.15.47:0/1884409331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c100ad0 0x7fae9c100ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009900 tx=0x7fae8c02a410 comp rx=0 tx=0).stop 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 -- 172.21.15.47:0/1884409331 shutdown_connections 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 --2- 172.21.15.47:0/1884409331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c102690 0x7fae9c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 --2- 172.21.15.47:0/1884409331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae9c101cd0 0x7fae9c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 --2- 172.21.15.47:0/1884409331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c100ad0 0x7fae9c100ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 -- 172.21.15.47:0/1884409331 >> 172.21.15.47:0/1884409331 conn(0x7fae9c0fc280 msgr2=0x7fae9c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 -- 172.21.15.47:0/1884409331 shutdown_connections 2024-09-10T09:56:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.112+0000 7faea0f1b640 1 -- 172.21.15.47:0/1884409331 wait complete. 2024-09-10T09:56:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.113+0000 7faea0f1b640 1 Processor -- start 2024-09-10T09:56:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.113+0000 7faea0f1b640 1 -- start start 2024-09-10T09:56:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7faea0f1b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae9c100ad0 0x7fae9c06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7faea0f1b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c101cd0 0x7fae9c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:11.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7faea0f1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c102690 0x7fae9c19f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7faea0f1b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c10c780 con 0x7fae9c102690 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7faea0f1b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c10c600 con 0x7fae9c101cd0 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7faea0f1b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c10c900 con 0x7fae9c100ad0 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c101cd0 0x7fae9c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c101cd0 0x7fae9c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41982/0 (socket says 172.21.15.47:41982) 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 -- 172.21.15.47:0/3653327771 learned_addr learned my addr 172.21.15.47:0/3653327771 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae9a59c640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae9c100ad0 0x7fae9c06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae9ad9d640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c102690 0x7fae9c19f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 -- 172.21.15.47:0/3653327771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae9c100ad0 msgr2=0x7fae9c06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae9c100ad0 0x7fae9c06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 -- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c102690 msgr2=0x7fae9c19f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c102690 0x7fae9c19f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 -- 172.21.15.47:0/3653327771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c19fb50 con 0x7fae9c101cd0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae99d9b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c101cd0 0x7fae9c19cc20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c920 tx=0x7fae9000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7fae8b7fe640 1 -- 172.21.15.47:0/3653327771 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90007c70 con 0x7fae9c101cd0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.114+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9c19fde0 con 0x7fae9c101cd0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.115+0000 7fae8b7fe640 1 -- 172.21.15.47:0/3653327771 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae90007e10 con 0x7fae9c101cd0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.115+0000 7fae8b7fe640 1 -- 172.21.15.47:0/3653327771 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90018890 con 0x7fae9c101cd0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.115+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9c1a02e0 con 0x7fae9c101cd0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.116+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9c107e70 con 0x7fae9c101cd0 2024-09-10T09:56:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.118+0000 7fae8b7fe640 1 -- 172.21.15.47:0/3653327771 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fae90018a30 con 0x7fae9c101cd0 2024-09-10T09:56:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.119+0000 7fae8b7fe640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae840a2bf0 0x7fae840a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.119+0000 7fae8b7fe640 1 -- 172.21.15.47:0/3653327771 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1361+0+0 (secure 0 0 0) 0x7fae900cd630 con 0x7fae9c101cd0 2024-09-10T09:56:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.119+0000 7fae9a59c640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae840a2bf0 0x7fae840a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:11.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.119+0000 7fae9a59c640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae840a2bf0 0x7fae840a50b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae8c02aa00 tx=0x7fae8c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:11.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.121+0000 7fae8b7fe640 1 -- 172.21.15.47:0/3653327771 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae90014030 con 0x7fae9c101cd0 2024-09-10T09:56:11.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:11.304+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fae9c062630 con 0x7fae840a2bf0 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:11.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:56:11.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:11 smithi162 ceph-mon[34963]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:56:11.651 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0[44916]: 2024-09-10T09:56:11.313+0000 7f1c99fcd640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T09:56:11.651 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0[44916]: 2024-09-10T09:56:11.316+0000 7f1c96607640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:56:11.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:56:11.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:56:11.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:11 smithi045 ceph-mon[31114]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:56:11.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: from='osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:56:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:11 smithi047 ceph-mon[35244]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:56:12.610 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:12.610 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi045 2024-09-10T09:56:12.610 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi045 to 17092M 2024-09-10T09:56:12.610 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:56:12.611 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.611 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:56:12.611 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.611 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:56:12.611 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:12.611 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi045 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi045 to 17092M 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi045 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi045 to 17092M 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:13.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:13 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:13 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:13 smithi162 ceph-mon[34963]: osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] boot 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:13 smithi162 ceph-mon[34963]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:13 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:13 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:13 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:13 smithi045 ceph-mon[31114]: osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] boot 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:13 smithi045 ceph-mon[31114]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:56:13.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:13.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:13 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:56:13.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:13 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:56:13.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:13 smithi047 ceph-mon[35244]: osd.0 [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] boot 2024-09-10T09:56:13.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:13 smithi047 ceph-mon[35244]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:56:13.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:13 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/3076125175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]: dispatch 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]: dispatch 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]': finished 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:14 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/1786558620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:56:14.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/3076125175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]: dispatch 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]: dispatch 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]': finished 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:14 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/1786558620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:56:14.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:56:14.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/3076125175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]: dispatch 2024-09-10T09:56:14.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]: dispatch 2024-09-10T09:56:14.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20cd1eda-0149-4eab-a5ff-9463c2b556fa"}]': finished 2024-09-10T09:56:14.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:56:14.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:14.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:14 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/1786558620' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:56:16.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:16 smithi162 ceph-mon[34963]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:16.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:16 smithi045 ceph-mon[31114]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:16.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:16 smithi047 ceph-mon[35244]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:18.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:18 smithi162 ceph-mon[34963]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:18.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:18 smithi045 ceph-mon[31114]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:18.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:18 smithi047 ceph-mon[35244]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:20.531 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:20 smithi047 ceph-mon[35244]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:20.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:20 smithi162 ceph-mon[34963]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:20.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:20 smithi045 ceph-mon[31114]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:22.618 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:22 smithi047 ceph-mon[35244]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:22.619 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:56:22.619 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:22 smithi162 ceph-mon[34963]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:56:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:22 smithi045 ceph-mon[31114]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:56:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:23.593 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:23 smithi047 ceph-mon[35244]: Deploying daemon osd.1 on smithi047 2024-09-10T09:56:23.604 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:23 smithi045 ceph-mon[31114]: Deploying daemon osd.1 on smithi047 2024-09-10T09:56:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:23 smithi162 ceph-mon[34963]: Deploying daemon osd.1 on smithi047 2024-09-10T09:56:24.596 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:24 smithi047 ceph-mon[35244]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:24.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:24 smithi162 ceph-mon[34963]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:24.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:24 smithi045 ceph-mon[31114]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:26 smithi162 ceph-mon[34963]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:26.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:26 smithi045 ceph-mon[31114]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:26.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:26 smithi047 ceph-mon[35244]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:28 smithi162 ceph-mon[34963]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:28.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:28 smithi045 ceph-mon[31114]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:28 smithi047 ceph-mon[35244]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:29.566 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:29.566 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:29.566 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:29.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:29.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:30.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:30 smithi047 ceph-mon[35244]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:30.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:30 smithi162 ceph-mon[34963]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:30.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:30 smithi045 ceph-mon[31114]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:32.361 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:32 smithi047 ceph-mon[35244]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:32.361 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.361 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.361 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:32.361 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:32.362 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:32 smithi162 ceph-mon[34963]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:32 smithi045 ceph-mon[31114]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:32.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:32.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.651+0000 7fae8b7fe640 1 -- 172.21.15.47:0/3653327771 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae9c062630 con 0x7fae840a2bf0 2024-09-10T09:56:32.653 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 1 on host 'smithi047' 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae840a2bf0 msgr2=0x7fae840a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae840a2bf0 0x7fae840a50b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae8c02aa00 tx=0x7fae8c0315b0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c101cd0 msgr2=0x7fae9c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c101cd0 0x7fae9c19cc20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c920 tx=0x7fae9000cde0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 shutdown_connections 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae840a2bf0 0x7fae840a50b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c102690 0x7fae9c19f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae9c101cd0 0x7fae9c19cc20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 --2- 172.21.15.47:0/3653327771 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae9c100ad0 0x7fae9c06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 >> 172.21.15.47:0/3653327771 conn(0x7fae9c0fc280 msgr2=0x7fae9c105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:32.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.653+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 shutdown_connections 2024-09-10T09:56:32.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:56:32.654+0000 7faea0f1b640 1 -- 172.21.15.47:0/3653327771 wait complete. 2024-09-10T09:56:33.331 DEBUG:teuthology.orchestra.run.smithi047:osd.1> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T09:56:33.333 INFO:tasks.cephadm:Deploying osd.2 on smithi162 with /dev/vg_nvme/lv_4... 2024-09-10T09:56:33.333 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:56:33.357 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:33 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:33.357 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:33 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:33.357 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:33 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:33.361 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:33 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:33.361 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:33 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:33.361 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:33 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:33.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:33 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:33.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:33 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:33.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:33 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:33.584 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:56:34.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:34 smithi162 ceph-mon[34963]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:34 smithi045 ceph-mon[31114]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:34.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:34 smithi047 ceph-mon[35244]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:35.109 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:56:34 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1[39724]: 2024-09-10T09:56:34.762+0000 7f951fe54640 -1 osd.1 0 log_to_monitors true 2024-09-10T09:56:35.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:35 smithi162 ceph-mon[34963]: from='osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:56:35.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:35 smithi162 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:56:35.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:35 smithi045 ceph-mon[31114]: from='osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:56:35.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:35 smithi045 ceph-mon[31114]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:56:35.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:35 smithi047 ceph-mon[35244]: from='osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:56:35.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:35 smithi047 ceph-mon[35244]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:56:35.953 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:56:35.978 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_4 2024-09-10T09:56:36.268 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:56:36.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:36.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:36.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:36.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:56:36.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:56:36.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:36.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.409 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1[39724]: 2024-09-10T09:56:36.121+0000 7f951b5b4640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T09:56:36.409 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1[39724]: 2024-09-10T09:56:36.125+0000 7f9517bee640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:36.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:36.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:36.729 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3308313324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 msgr2=0x7fa6e0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3308313324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0102000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009900 tx=0x7fa6d402a410 comp rx=0 tx=0).stop 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3308313324 shutdown_connections 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3308313324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6e01037c0 0x7fa6e010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3308313324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6e0102e00 0x7fa6e0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3308313324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0102000 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3308313324 >> 172.21.15.162:0/3308313324 conn(0x7fa6e00fd390 msgr2=0x7fa6e00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3308313324 shutdown_connections 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.703+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3308313324 wait complete. 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 Processor -- start 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 -- start start 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6e0102e00 0x7fa6e00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6e01037c0 0x7fa6e0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:36.730 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010d8e0 con 0x7fa6e01037c0 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010d760 con 0x7fa6e0102e00 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6e48d9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e010da60 con 0x7fa6e0101c00 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6de59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6de59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:57018/0 (socket says 172.21.15.162:57018) 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6de59c640 1 -- 172.21.15.162:0/3847180048 learned_addr learned my addr 172.21.15.162:0/3847180048 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.704+0000 7fa6ddd9b640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6e0102e00 0x7fa6e00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6ded9d640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6e01037c0 0x7fa6e0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6de59c640 1 -- 172.21.15.162:0/3847180048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6e0102e00 msgr2=0x7fa6e00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6de59c640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6e0102e00 0x7fa6e00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6de59c640 1 -- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6e01037c0 msgr2=0x7fa6e0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6de59c640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6e01037c0 0x7fa6e0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6de59c640 1 -- 172.21.15.162:0/3847180048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e00753e0 con 0x7fa6e0101c00 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6ded9d640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6e01037c0 0x7fa6e0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:56:36.731 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6de59c640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6d402aa00 tx=0x7fa6d4032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6cf7fe640 1 -- 172.21.15.162:0/3847180048 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d403f030 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6cf7fe640 1 -- 172.21.15.162:0/3847180048 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d4003ce0 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e01a2060 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6cf7fe640 1 -- 172.21.15.162:0/3847180048 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4035070 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.705+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e01a2510 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.706+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e0109020 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.707+0000 7fa6cf7fe640 1 -- 172.21.15.162:0/3847180048 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa6d4042030 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.708+0000 7fa6cf7fe640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa6c80a2af0 0x7fa6c80a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.708+0000 7fa6cf7fe640 1 -- 172.21.15.162:0/3847180048 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fa6d40eddf0 con 0x7fa6e0101c00 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.708+0000 7fa6ddd9b640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa6c80a2af0 0x7fa6c80a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.709+0000 7fa6ddd9b640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa6c80a2af0 0x7fa6c80a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0075d10 tx=0x7fa6d000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:36.732 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.712+0000 7fa6cf7fe640 1 -- 172.21.15.162:0/3847180048 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6d40bc6c0 con 0x7fa6e0101c00 2024-09-10T09:56:36.886 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:36.885+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa6e0062630 con 0x7fa6c80a2af0 2024-09-10T09:56:37.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi047 2024-09-10T09:56:37.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-10T09:56:37.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:56:37.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:56:37.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:37.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:37.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:56:37.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:56:37.390 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:37.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi047 2024-09-10T09:56:37.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-10T09:56:37.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:56:37.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:56:37.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:37.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:37.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:56:37.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:56:37.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:37.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi047 2024-09-10T09:56:37.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi047 to 17092M 2024-09-10T09:56:37.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:56:37.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:56:37.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:37.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:37.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:56:37.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:56:37.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:38.341 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:56:38.341 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:56:38.341 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:38.341 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:56:38.341 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:38.341 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] boot 2024-09-10T09:56:38.342 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:56:38.342 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:38.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:56:38.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:56:38.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:38.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:56:38.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:38.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] boot 2024-09-10T09:56:38.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:56:38.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:38.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:56:38.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:56:38.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:56:38.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:56:38.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:38.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: osd.1 [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] boot 2024-09-10T09:56:38.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:56:38.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:56:39.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:39 smithi162 ceph-mon[34963]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:56:39.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:39 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/3991916790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]: dispatch 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:39 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]: dispatch 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:39 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]': finished 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:39 smithi162 ceph-mon[34963]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:39 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:39 smithi045 ceph-mon[31114]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:39 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/3991916790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]: dispatch 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:39 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]: dispatch 2024-09-10T09:56:39.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:39 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]': finished 2024-09-10T09:56:39.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:39 smithi045 ceph-mon[31114]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:56:39.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:39 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:56:39.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:39 smithi047 ceph-mon[35244]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:56:39.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:39 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/3991916790' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]: dispatch 2024-09-10T09:56:39.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:39 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]: dispatch 2024-09-10T09:56:39.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:39 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5"}]': finished 2024-09-10T09:56:39.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:39 smithi047 ceph-mon[35244]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:56:39.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:39 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:56:40.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:40 smithi047 ceph-mon[35244]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:40.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:40 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/889224447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:56:40.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:40 smithi162 ceph-mon[34963]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:40.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:40 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/889224447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:56:40.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:40 smithi045 ceph-mon[31114]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:40.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:40 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/889224447' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:56:42.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:42 smithi047 ceph-mon[35244]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:42.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:42 smithi162 ceph-mon[34963]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:42.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:42 smithi045 ceph-mon[31114]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:44.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:44 smithi162 ceph-mon[34963]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:44.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:44 smithi047 ceph-mon[35244]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:44.452 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:44 smithi045 ceph-mon[31114]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:46.351 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:46 smithi162 ceph-mon[34963]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:46.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:46 smithi045 ceph-mon[31114]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:46.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:46 smithi047 ceph-mon[35244]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:48 smithi162 ceph-mon[34963]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:56:48.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:48.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:48 smithi162 ceph-mon[34963]: Deploying daemon osd.2 on smithi162 2024-09-10T09:56:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:48 smithi045 ceph-mon[31114]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:56:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:48 smithi045 ceph-mon[31114]: Deploying daemon osd.2 on smithi162 2024-09-10T09:56:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:48 smithi047 ceph-mon[35244]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:56:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:48.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:48 smithi047 ceph-mon[35244]: Deploying daemon osd.2 on smithi162 2024-09-10T09:56:50.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:50 smithi162 ceph-mon[34963]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:50.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:50 smithi045 ceph-mon[31114]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:50.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:50 smithi047 ceph-mon[35244]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:52 smithi162 ceph-mon[34963]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:52.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:52 smithi045 ceph-mon[31114]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:52.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:52 smithi047 ceph-mon[35244]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:54.314 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:54 smithi162 ceph-mon[34963]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:54.314 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:54 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:54.314 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:54 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:54.314 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:54 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:54.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:54 smithi045 ceph-mon[31114]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:54.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:54 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:54.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:54 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:54.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:54 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:54.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:54 smithi047 ceph-mon[35244]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:54.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:54 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:54.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:54 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:54.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:54 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.645 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:56 smithi162 ceph-mon[34963]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:56.645 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.645 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.645 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:56.645 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:56.645 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:56 smithi045 ceph-mon[31114]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:56 smithi047 ceph-mon[35244]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:57.211 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.208+0000 7fa6cf7fe640 1 -- 172.21.15.162:0/3847180048 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa6e0062630 con 0x7fa6c80a2af0 2024-09-10T09:56:57.211 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 2 on host 'smithi162' 2024-09-10T09:56:57.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa6c80a2af0 msgr2=0x7fa6c80a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:57.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa6c80a2af0 0x7fa6c80a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0075d10 tx=0x7fa6d000a2d0 comp rx=0 tx=0).stop 2024-09-10T09:56:57.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 msgr2=0x7fa6e0075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:57.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0075f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6d402aa00 tx=0x7fa6d4032da0 comp rx=0 tx=0).stop 2024-09-10T09:56:57.213 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 shutdown_connections 2024-09-10T09:56:57.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa6c80a2af0 0x7fa6c80a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:57.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6e01037c0 0x7fa6e0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:57.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6e0102e00 0x7fa6e00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:57.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 --2- 172.21.15.162:0/3847180048 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa6e0101c00 0x7fa6e0075f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:57.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 >> 172.21.15.162:0/3847180048 conn(0x7fa6e00fd390 msgr2=0x7fa6e01061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:57.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 shutdown_connections 2024-09-10T09:56:57.214 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:56:57.211+0000 7fa6e48d9640 1 -- 172.21.15.162:0/3847180048 wait complete. 2024-09-10T09:56:57.809 DEBUG:teuthology.orchestra.run.smithi162:osd.2> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T09:56:57.811 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2024-09-10T09:56:57.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:56:58.079 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:58.326 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:58 smithi162 ceph-mon[34963]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:58.326 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:58.326 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:58.326 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:56:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:58.462 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:58 smithi047 ceph-mon[35244]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:58.462 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:58.462 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:58.462 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:56:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:58.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:58 smithi045 ceph-mon[31114]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:56:58.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:58.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:56:58.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:56:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:00.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:56:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2[39954]: 2024-09-10T09:56:59.893+0000 7f448a9da640 -1 osd.2 0 log_to_monitors true 2024-09-10T09:57:00.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:00 smithi045 ceph-mon[31114]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:00.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:00 smithi045 ceph-mon[31114]: from='osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:57:00.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:00 smithi045 ceph-mon[31114]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:57:00.456 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:00 smithi162 ceph-mon[34963]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:00.456 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:00 smithi162 ceph-mon[34963]: from='osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:57:00.456 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:00 smithi162 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:57:00.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:00 smithi047 ceph-mon[35244]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:00.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:00 smithi047 ceph-mon[35244]: from='osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:57:00.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:00 smithi047 ceph-mon[35244]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:57:00.841 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:57:00.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-09-10T09:57:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:01.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:57:01.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:57:01.515 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:01.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1860594277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d8102660 msgr2=0x7fb6d8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1860594277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d8102660 0x7fb6d8104a50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009ca0 tx=0x7fb6c802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1860594277 shutdown_connections 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1860594277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d81078c0 0x7fb6d8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1860594277 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8104f90 0x7fb6d8107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1860594277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d8102660 0x7fb6d8104a50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1860594277 >> 172.21.15.45:0/1860594277 conn(0x7fb6d80fc280 msgr2=0x7fb6d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1860594277 shutdown_connections 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1860594277 wait complete. 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.601+0000 7fb6ddbce640 1 Processor -- start 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6ddbce640 1 -- start start 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6ddbce640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8102660 0x7fb6d8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6ddbce640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d8104f90 0x7fb6d81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6ddbce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d81078c0 0x7fb6d819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6ddbce640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d810d9a0 con 0x7fb6d81078c0 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6ddbce640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d810d820 con 0x7fb6d8104f90 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6ddbce640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d810db20 con 0x7fb6d8102660 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d81078c0 0x7fb6d819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d81078c0 0x7fb6d819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54176/0 (socket says 172.21.15.45:54176) 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 -- 172.21.15.45:0/1316811134 learned_addr learned my addr 172.21.15.45:0/1316811134 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d6ffd640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d8104f90 0x7fb6d81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d77fe640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8102660 0x7fb6d8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 -- 172.21.15.45:0/1316811134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8102660 msgr2=0x7fb6d8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8102660 0x7fb6d8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 -- 172.21.15.45:0/1316811134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d8104f90 msgr2=0x7fb6d81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d8104f90 0x7fb6d81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 -- 172.21.15.45:0/1316811134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d819fcb0 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.602+0000 7fb6d7fff640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d81078c0 0x7fb6d819f640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009fd0 tx=0x7fb6d000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.603+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0015020 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.603+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d0011d10 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.603+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d819ffa0 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.603+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000f2e0 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.603+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d81a04d0 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.604+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d80fdd70 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.605+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fb6d0018020 con 0x7fb6d81078c0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.606+0000 7fb6d4ff9640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb6c40a2ba0 0x7fb6c40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.606+0000 7fb6d77fe640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb6c40a2ba0 0x7fb6c40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.606+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fb6d00cd110 con 0x7fb6d81078c0 2024-09-10T09:57:01.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.607+0000 7fb6d77fe640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb6c40a2ba0 0x7fb6c40a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000f80 tx=0x7fb6c8031bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:01.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.610+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6d001f020 con 0x7fb6d81078c0 2024-09-10T09:57:01.649 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2[39954]: 2024-09-10T09:57:01.189+0000 7f448613a640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T09:57:01.649 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2[39954]: 2024-09-10T09:57:01.193+0000 7f4482774640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:01.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:01.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:01.794+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb6d8062630 con 0x7fb6c40a2ba0 2024-09-10T09:57:02.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:02.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi162 2024-09-10T09:57:02.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi162 to 17088M 2024-09-10T09:57:02.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:57:02.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:57:02.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:02.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:02.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:02.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:02.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:02.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi162 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi162 to 17088M 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi162 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi162 to 17088M 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:02.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:03.483 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:03 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:57:03.483 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:03 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:57:03.484 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:03 smithi045 ceph-mon[31114]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:03.484 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:03 smithi045 ceph-mon[31114]: osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] boot 2024-09-10T09:57:03.484 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:03 smithi045 ceph-mon[31114]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:57:03.484 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:03 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:03.484 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:03 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:57:03.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:03 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:57:03.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:03 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:57:03.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:03 smithi162 ceph-mon[34963]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:03.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:03 smithi162 ceph-mon[34963]: osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] boot 2024-09-10T09:57:03.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:03 smithi162 ceph-mon[34963]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:57:03.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:03 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:03.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:03 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:57:03.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:03 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:57:03.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:03 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:57:03.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:03 smithi047 ceph-mon[35244]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:03.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:03 smithi047 ceph-mon[35244]: osd.2 [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] boot 2024-09-10T09:57:03.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:03 smithi047 ceph-mon[35244]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:57:03.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:03 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:57:03.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:03 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/971956608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14f9d3be-6d90-4490-b946-4a96c46736f9"}]: dispatch 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/971956608' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14f9d3be-6d90-4490-b946-4a96c46736f9"}]': finished 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T09:57:04.508 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:04.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/971956608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14f9d3be-6d90-4490-b946-4a96c46736f9"}]: dispatch 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/971956608' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14f9d3be-6d90-4490-b946-4a96c46736f9"}]': finished 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T09:57:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:04.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/971956608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14f9d3be-6d90-4490-b946-4a96c46736f9"}]: dispatch 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/971956608' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14f9d3be-6d90-4490-b946-4a96c46736f9"}]': finished 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T09:57:04.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:04.761 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:04 smithi045 sudo[49801]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:57:04.762 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:04 smithi045 sudo[49801]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:04.762 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:04 smithi045 sudo[49801]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:04.762 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:04 smithi045 sudo[49801]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:05.210+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb6d0097730 con 0x7fb6d81078c0 2024-09-10T09:57:05.470 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:05 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2252395131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:05.470 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:05 smithi047 ceph-mon[35244]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T09:57:05.470 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:05 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:05.470 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:05 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:05.471 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43903]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:57:05.471 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43903]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:05.471 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43903]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:05.471 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43903]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:05.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:05 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2252395131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:05.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:05 smithi162 ceph-mon[34963]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T09:57:05.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:05 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:05.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:05 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:05.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:05 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2252395131' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:05.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:05 smithi045 ceph-mon[31114]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T09:57:05.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:05 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:05.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:05 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:05.651 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:05 smithi045 sudo[49853]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:57:05.651 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:05 smithi045 sudo[49853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:05.651 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:05 smithi045 sudo[49853]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:05.651 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 09:57:05 smithi045 sudo[49853]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:06.160 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43906]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:57:06.160 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43906]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:06.160 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43906]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:06.160 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 09:57:05 smithi047 sudo[43906]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:06.248 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:06 smithi162 ceph-mon[34963]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:06.248 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:06 smithi162 ceph-mon[34963]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:57:06.249 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:06.249 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:05 smithi162 sudo[43596]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:57:06.249 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:05 smithi162 sudo[43596]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:06.249 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:05 smithi162 sudo[43596]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:06.249 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:06 smithi162 sudo[43596]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:06.649 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:06 smithi162 sudo[43599]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:57:06.650 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:06 smithi162 sudo[43599]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:06.650 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:06 smithi162 sudo[43599]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:06.650 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 09:57:06 smithi162 sudo[43599]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:06 smithi045 ceph-mon[31114]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:06 smithi045 ceph-mon[31114]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:57:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:06.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:06 smithi047 ceph-mon[35244]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:06.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:06 smithi047 ceph-mon[35244]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:57:06.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:07.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:06 smithi045 sudo[50235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:57:07.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:06 smithi045 sudo[50235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:07.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:06 smithi045 sudo[50235]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:07.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:06 smithi045 sudo[50235]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:07 smithi045 sudo[50427]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:57:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:07 smithi045 sudo[50427]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:07 smithi045 sudo[50427]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:07 smithi045 sudo[50427]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:07.669 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:57:07.669 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:07.669 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43602]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:07.669 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43602]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:57:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43605]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:07 smithi162 sudo[43605]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:08.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:07 smithi047 sudo[43909]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:57:08.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:07 smithi047 sudo[43909]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:08.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:07 smithi047 sudo[43909]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:08.201 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:07 smithi047 sudo[43909]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:08.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:08.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 sudo[43912]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 sudo[43912]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 sudo[43912]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 sudo[43912]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:09.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:09.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:09.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:09.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:09 smithi045 ceph-mon[31114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:09.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:09.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:09.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:09.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:09.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:09.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:09.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:09.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:09 smithi162 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:57:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:57:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:57:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:57:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:09 smithi047 ceph-mon[35244]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:57:10.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:10 smithi045 ceph-mon[31114]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:10.534 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:10 smithi162 ceph-mon[34963]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:10.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:10 smithi047 ceph-mon[35244]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:12.574 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:12 smithi045 ceph-mon[31114]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:12 smithi162 ceph-mon[34963]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:12.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:12 smithi047 ceph-mon[35244]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:13.352 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:57:13.352 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:13.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:13 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:57:13.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:13 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:13.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:13 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:57:13.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:13 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:14.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:14 smithi045 ceph-mon[31114]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:14.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:14 smithi045 ceph-mon[31114]: Deploying daemon osd.3 on smithi045 2024-09-10T09:57:14.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:14 smithi162 ceph-mon[34963]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:14.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:14 smithi162 ceph-mon[34963]: Deploying daemon osd.3 on smithi045 2024-09-10T09:57:14.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:14 smithi047 ceph-mon[35244]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:14.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:14 smithi047 ceph-mon[35244]: Deploying daemon osd.3 on smithi045 2024-09-10T09:57:16.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:16 smithi162 ceph-mon[34963]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-10T09:57:16.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:16 smithi045 ceph-mon[31114]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-10T09:57:16.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:16 smithi047 ceph-mon[35244]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-10T09:57:18.387 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:18 smithi045 ceph-mon[31114]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-10T09:57:18.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:18 smithi162 ceph-mon[34963]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-10T09:57:18.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:18 smithi047 ceph-mon[35244]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-10T09:57:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:19.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:19.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:19.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:19.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:20.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:20 smithi162 ceph-mon[34963]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T09:57:20.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:20 smithi045 ceph-mon[31114]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T09:57:20.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:20 smithi047 ceph-mon[35244]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-10T09:57:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:22 smithi162 ceph-mon[34963]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:22 smithi045 ceph-mon[31114]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:22.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:22.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:22 smithi047 ceph-mon[35244]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:22.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.709+0000 7fb6d4ff9640 1 -- 172.21.15.45:0/1316811134 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb6d8062630 con 0x7fb6c40a2ba0 2024-09-10T09:57:22.711 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb6c40a2ba0 msgr2=0x7fb6c40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb6c40a2ba0 0x7fb6c40a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000f80 tx=0x7fb6c8031bd0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d81078c0 msgr2=0x7fb6d819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d81078c0 0x7fb6d819f640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009fd0 tx=0x7fb6d000b820 comp rx=0 tx=0).stop 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 shutdown_connections 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb6c40a2ba0 0x7fb6c40a5060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6d81078c0 0x7fb6d819f640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb6d8104f90 0x7fb6d81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 --2- 172.21.15.45:0/1316811134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6d8102660 0x7fb6d8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 >> 172.21.15.45:0/1316811134 conn(0x7fb6d80fc280 msgr2=0x7fb6d8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:22.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 shutdown_connections 2024-09-10T09:57:22.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:57:22.712+0000 7fb6ddbce640 1 -- 172.21.15.45:0/1316811134 wait complete. 2024-09-10T09:57:23.370 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T09:57:23.372 INFO:tasks.cephadm:Deploying osd.4 on smithi047 with /dev/vg_nvme/lv_3... 2024-09-10T09:57:23.373 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:57:23.630 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:57:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:23 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:23 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:23 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:23 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:23 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:23 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:23 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:23 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:23 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:24.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:24 smithi162 ceph-mon[34963]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:24.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:24 smithi045 ceph-mon[31114]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:24.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:24 smithi047 ceph-mon[35244]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:25.224 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 09:57:25 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3[51844]: 2024-09-10T09:57:25.016+0000 7f444178c640 -1 osd.3 0 log_to_monitors true 2024-09-10T09:57:25.359 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:25 smithi047 ceph-mon[35244]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:57:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:25 smithi162 ceph-mon[34963]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:57:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:25 smithi045 ceph-mon[31114]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:57:26.258 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:57:26.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-09-10T09:57:26.539 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:57:26.587 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:26 smithi047 ceph-mon[35244]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:26.588 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:26 smithi047 ceph-mon[35244]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:57:26.588 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:26 smithi047 ceph-mon[35244]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:57:26.588 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:26 smithi047 ceph-mon[35244]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:57:26.588 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:26 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:26.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:26 smithi162 ceph-mon[34963]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:26.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:26 smithi162 ceph-mon[34963]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:57:26.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:26 smithi162 ceph-mon[34963]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:57:26.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:26 smithi162 ceph-mon[34963]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:57:26.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:26 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:26.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:26 smithi045 ceph-mon[31114]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T09:57:26.654 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:26 smithi045 ceph-mon[31114]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:57:26.654 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:26 smithi045 ceph-mon[31114]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:57:26.654 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:26 smithi045 ceph-mon[31114]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:57:26.654 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:26 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:26.654 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 09:57:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3[51844]: 2024-09-10T09:57:26.353+0000 7f443ceec640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T09:57:26.654 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 09:57:26 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3[51844]: 2024-09-10T09:57:26.360+0000 7f4439526640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:57:27.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 -- 172.21.15.47:0/3275936390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08100ad0 msgr2=0x7f4e08100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:27.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 --2- 172.21.15.47:0/3275936390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08100ad0 0x7f4e08100ed0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009900 tx=0x7f4df802a410 comp rx=0 tx=0).stop 2024-09-10T09:57:27.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 -- 172.21.15.47:0/3275936390 shutdown_connections 2024-09-10T09:57:27.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 --2- 172.21.15.47:0/3275936390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e08102690 0x7f4e08108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:27.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 --2- 172.21.15.47:0/3275936390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08101cd0 0x7f4e08102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 --2- 172.21.15.47:0/3275936390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08100ad0 0x7f4e08100ed0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 -- 172.21.15.47:0/3275936390 >> 172.21.15.47:0/3275936390 conn(0x7f4e080fc280 msgr2=0x7f4e080fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 -- 172.21.15.47:0/3275936390 shutdown_connections 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.159+0000 7f4e0c975640 1 -- 172.21.15.47:0/3275936390 wait complete. 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.160+0000 7f4e0c975640 1 Processor -- start 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.160+0000 7f4e0c975640 1 -- start start 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0c975640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08100ad0 0x7f4e0806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0c975640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08101cd0 0x7f4e0819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0c975640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e08102690 0x7f4e0819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0c975640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0810c780 con 0x7f4e08100ad0 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0c975640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0810c600 con 0x7f4e08101cd0 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0c975640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0810c900 con 0x7f4e08102690 2024-09-10T09:57:27.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08100ad0 0x7f4e0806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:27.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e05d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08101cd0 0x7f4e0819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:27.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08100ad0 0x7f4e0806aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.47:47382/0 (socket says 172.21.15.47:47382) 2024-09-10T09:57:27.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 -- 172.21.15.47:0/2554577971 learned_addr learned my addr 172.21.15.47:0/2554577971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-10T09:57:27.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e06d9d640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e08102690 0x7f4e0819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 -- 172.21.15.47:0/2554577971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e08102690 msgr2=0x7f4e0819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e08102690 0x7f4e0819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 -- 172.21.15.47:0/2554577971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08101cd0 msgr2=0x7f4e0819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08101cd0 0x7f4e0819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 -- 172.21.15.47:0/2554577971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0819fbd0 con 0x7f4e08100ad0 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e05d9b640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08101cd0 0x7f4e0819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.161+0000 7f4e0659c640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08100ad0 0x7f4e0806aad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4df8031440 tx=0x7f4df8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.162+0000 7f4dff7fe640 1 -- 172.21.15.47:0/2554577971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df803f030 con 0x7f4e08100ad0 2024-09-10T09:57:27.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.162+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0819fe00 con 0x7f4e08100ad0 2024-09-10T09:57:27.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.162+0000 7f4dff7fe640 1 -- 172.21.15.47:0/2554577971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df8003ce0 con 0x7f4e08100ad0 2024-09-10T09:57:27.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.162+0000 7f4dff7fe640 1 -- 172.21.15.47:0/2554577971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8035070 con 0x7f4e08100ad0 2024-09-10T09:57:27.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.162+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e081a0260 con 0x7f4e08100ad0 2024-09-10T09:57:27.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.163+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e08107f20 con 0x7f4e08100ad0 2024-09-10T09:57:27.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.164+0000 7f4dff7fe640 1 -- 172.21.15.47:0/2554577971 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4df8042030 con 0x7f4e08100ad0 2024-09-10T09:57:27.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.165+0000 7f4dff7fe640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4df40a2c00 0x7f4df40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:27.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.165+0000 7f4dff7fe640 1 -- 172.21.15.47:0/2554577971 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f4df80ee2b0 con 0x7f4e08100ad0 2024-09-10T09:57:27.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.165+0000 7f4e05d9b640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4df40a2c00 0x7f4df40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:27.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.165+0000 7f4e05d9b640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4df40a2c00 0x7f4df40a50c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e0806b050 tx=0x7f4df0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:27.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.170+0000 7f4dff7fe640 1 -- 172.21.15.47:0/2554577971 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4df80bbe60 con 0x7f4e08100ad0 2024-09-10T09:57:27.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:27.469+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4e08062630 con 0x7f4df40a2c00 2024-09-10T09:57:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:57:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:57:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi045 2024-09-10T09:57:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi045 to 8546M 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi045 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi045 to 8546M 2024-09-10T09:57:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:27.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:27.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi045 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:27.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi045 to 8546M 2024-09-10T09:57:27.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:27.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:27.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:27.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] boot 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] boot 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:28.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:28.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: osd.3 [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] boot 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: from='client.14319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:29.375 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:29 smithi047 ceph-mon[35244]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:57:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:29 smithi162 ceph-mon[34963]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:57:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:29 smithi045 ceph-mon[31114]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:57:30.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:30 smithi047 ceph-mon[35244]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:30.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:30 smithi047 ceph-mon[35244]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:57:30.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:30 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/4127153796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]: dispatch 2024-09-10T09:57:30.553 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:30 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]: dispatch 2024-09-10T09:57:30.554 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:30 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]': finished 2024-09-10T09:57:30.554 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:30 smithi047 ceph-mon[35244]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T09:57:30.554 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:30 smithi162 ceph-mon[34963]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:30 smithi162 ceph-mon[34963]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:30 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/4127153796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]: dispatch 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:30 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]: dispatch 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:30 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]': finished 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:30 smithi162 ceph-mon[34963]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:30.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:30 smithi045 ceph-mon[31114]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:30.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:30 smithi045 ceph-mon[31114]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:57:30.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:30 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/4127153796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]: dispatch 2024-09-10T09:57:30.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:30 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]: dispatch 2024-09-10T09:57:30.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:30 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e9ec487-151e-4982-b522-3a2c3f90e407"}]': finished 2024-09-10T09:57:30.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:30 smithi045 ceph-mon[31114]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T09:57:30.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:31.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:31 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/388903289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:31 smithi162 ceph-mon[34963]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T09:57:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:31 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/388903289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:31 smithi045 ceph-mon[31114]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T09:57:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:31 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/388903289' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:31 smithi047 ceph-mon[35244]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T09:57:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:32 smithi162 ceph-mon[34963]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:32 smithi162 ceph-mon[34963]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:57:32.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:32.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:32 smithi045 ceph-mon[31114]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:32.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:32 smithi045 ceph-mon[31114]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:57:32.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:32 smithi047 ceph-mon[35244]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:32 smithi047 ceph-mon[35244]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:57:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:34.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:34 smithi162 ceph-mon[34963]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:34.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:34 smithi045 ceph-mon[31114]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:34.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:34 smithi047 ceph-mon[35244]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:36.525 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:36 smithi047 ceph-mon[35244]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-10T09:57:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:36 smithi162 ceph-mon[34963]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-10T09:57:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:36 smithi045 ceph-mon[31114]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-10T09:57:38.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:38 smithi162 ceph-mon[34963]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T09:57:38.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:57:38.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:38 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:38 smithi045 ceph-mon[31114]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T09:57:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:57:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:38 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:38.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:38 smithi047 ceph-mon[35244]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T09:57:38.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:57:38.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:38 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:39.574 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:39 smithi047 ceph-mon[35244]: Deploying daemon osd.4 on smithi047 2024-09-10T09:57:39.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:39 smithi162 ceph-mon[34963]: Deploying daemon osd.4 on smithi047 2024-09-10T09:57:39.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:39 smithi045 ceph-mon[31114]: Deploying daemon osd.4 on smithi047 2024-09-10T09:57:40.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:40 smithi162 ceph-mon[34963]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:57:40.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:40 smithi045 ceph-mon[31114]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:57:40.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:40 smithi047 ceph-mon[35244]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:57:42.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:42 smithi162 ceph-mon[34963]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T09:57:42.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:42 smithi045 ceph-mon[31114]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T09:57:42.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:42 smithi047 ceph-mon[35244]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T09:57:44.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:44 smithi162 ceph-mon[34963]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T09:57:44.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:44 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:44.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:44 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:44.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:44 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:44.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:44 smithi045 ceph-mon[31114]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T09:57:44.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:44 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:44.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:44 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:44.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:44 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:44.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:44 smithi047 ceph-mon[35244]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-10T09:57:44.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:44 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:44.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:44 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:44.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:44 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:46.478 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:46 smithi047 ceph-mon[35244]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:57:46.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:46 smithi162 ceph-mon[34963]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:57:46.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:46 smithi045 ceph-mon[31114]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:57:47.890 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:47 smithi047 ceph-mon[35244]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:47.890 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:47.890 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:47.891 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:47.891 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:47.891 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:47.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.931+0000 7f4dff7fe640 1 -- 172.21.15.47:0/2554577971 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4e08062630 con 0x7f4df40a2c00 2024-09-10T09:57:47.933 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 4 on host 'smithi047' 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4df40a2c00 msgr2=0x7f4df40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4df40a2c00 0x7f4df40a50c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e0806b050 tx=0x7f4df0005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08100ad0 msgr2=0x7f4e0806aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08100ad0 0x7f4e0806aad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4df8031440 tx=0x7f4df8032ce0 comp rx=0 tx=0).stop 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 shutdown_connections 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4df40a2c00 0x7f4df40a50c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4e08102690 0x7f4e0819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4e08101cd0 0x7f4e0819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 --2- 172.21.15.47:0/2554577971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e08100ad0 0x7f4e0806aad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:47.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 >> 172.21.15.47:0/2554577971 conn(0x7f4e080fc280 msgr2=0x7f4e081050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 shutdown_connections 2024-09-10T09:57:47.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:57:47.933+0000 7f4e0c975640 1 -- 172.21.15.47:0/2554577971 wait complete. 2024-09-10T09:57:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:47 smithi162 ceph-mon[34963]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:47 smithi045 ceph-mon[31114]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:48.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:48.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.547 DEBUG:teuthology.orchestra.run.smithi047:osd.4> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T09:57:48.549 INFO:tasks.cephadm:Deploying osd.5 on smithi162 with /dev/vg_nvme/lv_3... 2024-09-10T09:57:48.550 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:57:48.774 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:48.774 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.774 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.802 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:57:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:49.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:49.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:49.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:50.044 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:49 smithi047 ceph-mon[35244]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:49 smithi162 ceph-mon[34963]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:50.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:49 smithi045 ceph-mon[31114]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:50.784 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 09:57:50 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4[46695]: 2024-09-10T09:57:50.519+0000 7fadd5661640 -1 osd.4 0 log_to_monitors true 2024-09-10T09:57:51.132 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:50 smithi047 ceph-mon[35244]: from='osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:57:51.132 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:50 smithi047 ceph-mon[35244]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:57:51.233 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:50 smithi162 ceph-mon[34963]: from='osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:57:51.233 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:50 smithi162 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:57:51.235 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:57:51.260 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_3 2024-09-10T09:57:51.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:50 smithi045 ceph-mon[31114]: from='osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:57:51.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:50 smithi045 ceph-mon[31114]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:57:51.516 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:57:52.002 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 -- 172.21.15.162:0/3063647544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3320101c00 msgr2=0x7f3320102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 --2- 172.21.15.162:0/3063647544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3320101c00 0x7f3320102000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3310009900 tx=0x7f331002a410 comp rx=0 tx=0).stop 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 -- 172.21.15.162:0/3063647544 shutdown_connections 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 --2- 172.21.15.162:0/3063647544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33201037c0 0x7f332010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 --2- 172.21.15.162:0/3063647544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320102e00 0x7f3320103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 --2- 172.21.15.162:0/3063647544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3320101c00 0x7f3320102000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 -- 172.21.15.162:0/3063647544 >> 172.21.15.162:0/3063647544 conn(0x7f33200fd3b0 msgr2=0x7f33200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 -- 172.21.15.162:0/3063647544 shutdown_connections 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.998+0000 7f3326d53640 1 -- 172.21.15.162:0/3063647544 wait complete. 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 Processor -- start 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 -- start start 2024-09-10T09:57:52.003 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320101c00 0x7f3320075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3320102e00 0x7f33200745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33201037c0 0x7f3320074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f332010d8e0 con 0x7f3320101c00 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f332010d760 con 0x7f3320102e00 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3326d53640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f332010da60 con 0x7f33201037c0 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f33252f0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33201037c0 0x7f3320074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3324aef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320101c00 0x7f3320075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3324aef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320101c00 0x7f3320075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.162:54846/0 (socket says 172.21.15.162:54846) 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3324aef640 1 -- 172.21.15.162:0/2286923187 learned_addr learned my addr 172.21.15.162:0/2286923187 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:57:52.004 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f331ffff640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3320102e00 0x7f33200745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f33252f0640 1 -- 172.21.15.162:0/2286923187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3320102e00 msgr2=0x7f33200745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f33252f0640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3320102e00 0x7f33200745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f33252f0640 1 -- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320101c00 msgr2=0x7f3320075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f33252f0640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320101c00 0x7f3320075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f33252f0640 1 -- 172.21.15.162:0/2286923187 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33200753e0 con 0x7f33201037c0 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:51.999+0000 7f3324aef640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320101c00 0x7f3320075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.000+0000 7f33252f0640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33201037c0 0x7f3320074b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f331800b410 tx=0x7f331800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.000+0000 7f331dffb640 1 -- 172.21.15.162:0/2286923187 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3318015020 con 0x7f33201037c0 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.000+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33201a20b0 con 0x7f33201037c0 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.000+0000 7f331dffb640 1 -- 172.21.15.162:0/2286923187 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3318011d10 con 0x7f33201037c0 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.000+0000 7f331dffb640 1 -- 172.21.15.162:0/2286923187 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331800f2e0 con 0x7f33201037c0 2024-09-10T09:57:52.005 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.000+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33201a25f0 con 0x7f33201037c0 2024-09-10T09:57:52.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.001+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3320108fa0 con 0x7f33201037c0 2024-09-10T09:57:52.006 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.003+0000 7f331dffb640 1 -- 172.21.15.162:0/2286923187 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3318018020 con 0x7f33201037c0 2024-09-10T09:57:52.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.003+0000 7f331dffb640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f330c0a2b80 0x7f330c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:52.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.004+0000 7f331dffb640 1 -- 172.21.15.162:0/2286923187 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f33180cd610 con 0x7f33201037c0 2024-09-10T09:57:52.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.004+0000 7f3324aef640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f330c0a2b80 0x7f330c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:52.007 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.004+0000 7f3324aef640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f330c0a2b80 0x7f330c0a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3310000c00 tx=0x7f33100046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:52.014 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.012+0000 7f331dffb640 1 -- 172.21.15.162:0/2286923187 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f331801f020 con 0x7f33201037c0 2024-09-10T09:57:52.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:57:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:57:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:57:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:57:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi047 2024-09-10T09:57:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi047 to 8546M 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.194 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:57:52.192+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3320062630 con 0x7f330c0a2b80 2024-09-10T09:57:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:57:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:57:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi047 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi047 to 8546M 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:52.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.409 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4[46695]: 2024-09-10T09:57:51.945+0000 7fadd0dc1640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T09:57:52.409 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4[46695]: 2024-09-10T09:57:51.952+0000 7fadcd3fb640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi047 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:52.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:57:52.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi047 to 8546M 2024-09-10T09:57:52.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:52.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:52.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:52.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:52 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:57:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:57:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:53.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:53.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:53.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:52 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:53.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:57:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:57:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:57:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:57:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:57:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:52 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:54.231 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:53 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:57:54.232 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:53 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:57:54.232 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:53 smithi162 ceph-mon[34963]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:54.232 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:53 smithi162 ceph-mon[34963]: osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] boot 2024-09-10T09:57:54.232 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:53 smithi162 ceph-mon[34963]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:57:54.232 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:54.254 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:53 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:57:54.255 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:53 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:57:54.255 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:53 smithi047 ceph-mon[35244]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:54.255 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:53 smithi047 ceph-mon[35244]: osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] boot 2024-09-10T09:57:54.255 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:53 smithi047 ceph-mon[35244]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:57:54.255 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:53 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:57:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:53 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:57:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:53 smithi045 ceph-mon[31114]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T09:57:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:53 smithi045 ceph-mon[31114]: osd.4 [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] boot 2024-09-10T09:57:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:53 smithi045 ceph-mon[31114]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:57:54.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:57:55.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:54 smithi162 ceph-mon[34963]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:57:55.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:54 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/4041063830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]: dispatch 2024-09-10T09:57:55.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:54 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]: dispatch 2024-09-10T09:57:55.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:54 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]': finished 2024-09-10T09:57:55.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:54 smithi162 ceph-mon[34963]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:57:55.051 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:54 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:57:55.251 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:54 smithi045 ceph-mon[31114]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:57:55.251 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:54 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/4041063830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]: dispatch 2024-09-10T09:57:55.251 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:54 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]: dispatch 2024-09-10T09:57:55.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:54 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]': finished 2024-09-10T09:57:55.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:54 smithi045 ceph-mon[31114]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:57:55.252 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:54 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:57:55.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:54 smithi047 ceph-mon[35244]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:57:55.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:54 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/4041063830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]: dispatch 2024-09-10T09:57:55.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:54 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]: dispatch 2024-09-10T09:57:55.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:54 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "086c844a-fd5a-4ded-9156-a2ccd034521f"}]': finished 2024-09-10T09:57:55.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:54 smithi047 ceph-mon[35244]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:57:55.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:54 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:57:56.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:55 smithi045 ceph-mon[31114]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:57:56.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:55 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/1596838171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:56.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:55 smithi162 ceph-mon[34963]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:57:56.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:55 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/1596838171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:56.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:55 smithi047 ceph-mon[35244]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:57:56.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:55 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/1596838171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:57:58.212 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:57 smithi047 ceph-mon[35244]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:57:58.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:57 smithi162 ceph-mon[34963]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:57:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:57 smithi045 ceph-mon[31114]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:00.214 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:57:59 smithi162 ceph-mon[34963]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:00.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:57:59 smithi045 ceph-mon[31114]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:00.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:57:59 smithi047 ceph-mon[35244]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:02.249 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:01 smithi162 ceph-mon[34963]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:02.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:01 smithi045 ceph-mon[31114]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:02.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:01 smithi047 ceph-mon[35244]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:03.304 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:58:03.304 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:03.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:58:03.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:03.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:58:03.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:04.319 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:03 smithi162 ceph-mon[34963]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:04.319 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:03 smithi162 ceph-mon[34963]: Deploying daemon osd.5 on smithi162 2024-09-10T09:58:04.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:03 smithi045 ceph-mon[31114]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:04.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:03 smithi045 ceph-mon[31114]: Deploying daemon osd.5 on smithi162 2024-09-10T09:58:04.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:03 smithi047 ceph-mon[35244]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:04.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:03 smithi047 ceph-mon[35244]: Deploying daemon osd.5 on smithi162 2024-09-10T09:58:06.210 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:05 smithi162 ceph-mon[34963]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:06.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:05 smithi045 ceph-mon[31114]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:06.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:05 smithi047 ceph-mon[35244]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:08.280 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:07 smithi162 ceph-mon[34963]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:08.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:07 smithi045 ceph-mon[31114]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:08.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:07 smithi047 ceph-mon[35244]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:09.244 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:09.244 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:09.244 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:09.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:09.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:09.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:10.383 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:09 smithi162 ceph-mon[34963]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:10.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:09 smithi045 ceph-mon[31114]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:10.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:09 smithi047 ceph-mon[35244]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:12.673 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.671+0000 7f331dffb640 1 -- 172.21.15.162:0/2286923187 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3320062630 con 0x7f330c0a2b80 2024-09-10T09:58:12.673 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 5 on host 'smithi162' 2024-09-10T09:58:12.675 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f330c0a2b80 msgr2=0x7f330c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.675 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f330c0a2b80 0x7f330c0a5040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3310000c00 tx=0x7f33100046c0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.675 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33201037c0 msgr2=0x7f3320074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.675 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33201037c0 0x7f3320074b20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f331800b410 tx=0x7f331800b8d0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 shutdown_connections 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f330c0a2b80 0x7f330c0a5040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f33201037c0 0x7f3320074b20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3320102e00 0x7f33200745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 --2- 172.21.15.162:0/2286923187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3320101c00 0x7f3320075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 >> 172.21.15.162:0/2286923187 conn(0x7f33200fd3b0 msgr2=0x7f3320106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 shutdown_connections 2024-09-10T09:58:12.676 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:58:12.674+0000 7f3326d53640 1 -- 172.21.15.162:0/2286923187 wait complete. 2024-09-10T09:58:12.685 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:12 smithi162 ceph-mon[34963]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:12.685 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.685 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.685 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:12.685 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:12.685 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:12 smithi045 ceph-mon[31114]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:12.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:12.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:12.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:12 smithi047 ceph-mon[35244]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:12.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:12.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:12.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:12.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:13.251 DEBUG:teuthology.orchestra.run.smithi162:osd.5> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T09:58:13.254 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2024-09-10T09:58:13.254 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:58:13.506 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:13 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:13.506 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:13 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:13.506 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:13 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:13.525 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:13.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:13.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:13.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:13 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:13.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:13 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:13.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:13 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:13.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:13 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:14.576 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:14 smithi162 ceph-mon[34963]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:14.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:14 smithi045 ceph-mon[31114]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:14.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:14 smithi047 ceph-mon[35244]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:15.476 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 09:58:15 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T09:58:15.191+0000 7fd85b7b0640 -1 osd.5 0 log_to_monitors true 2024-09-10T09:58:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:15 smithi045 ceph-mon[31114]: from='osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:58:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:15 smithi045 ceph-mon[31114]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:58:15.880 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:15 smithi162 ceph-mon[34963]: from='osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:58:15.881 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:15 smithi162 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:58:15.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:15 smithi047 ceph-mon[35244]: from='osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:58:15.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:15 smithi047 ceph-mon[35244]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:58:16.051 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:58:16.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-09-10T09:58:16.330 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:16.616 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:16.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:16.618 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.805+0000 7f875de6c640 1 -- 172.21.15.45:0/582479843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758100ad0 msgr2=0x7f8758100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.805+0000 7f875de6c640 1 --2- 172.21.15.45:0/582479843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758100ad0 0x7f8758100ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f874802a410 comp rx=0 tx=0).stop 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 -- 172.21.15.45:0/582479843 shutdown_connections 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 --2- 172.21.15.45:0/582479843 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8758102690 0x7f8758108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 --2- 172.21.15.45:0/582479843 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8758101cd0 0x7f8758102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 --2- 172.21.15.45:0/582479843 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758100ad0 0x7f8758100ed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 -- 172.21.15.45:0/582479843 >> 172.21.15.45:0/582479843 conn(0x7f87580fc280 msgr2=0x7f87580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 -- 172.21.15.45:0/582479843 shutdown_connections 2024-09-10T09:58:16.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 -- 172.21.15.45:0/582479843 wait complete. 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 Processor -- start 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 -- start start 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8758100ad0 0x7f875806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758101cd0 0x7f875819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.806+0000 7f875de6c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8758102690 0x7f875819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f875de6c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f875810c730 con 0x7f8758101cd0 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f875de6c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f875810c5b0 con 0x7f8758100ad0 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f875de6c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f875810c8b0 con 0x7f8758102690 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758101cd0 0x7f875819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758101cd0 0x7f875819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42798/0 (socket says 172.21.15.45:42798) 2024-09-10T09:58:16.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 -- 172.21.15.45:0/299676764 learned_addr learned my addr 172.21.15.45:0/299676764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f87577fe640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8758100ad0 0x7f875806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f8757fff640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8758102690 0x7f875819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 -- 172.21.15.45:0/299676764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8758102690 msgr2=0x7f875819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8758102690 0x7f875819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 -- 172.21.15.45:0/299676764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8758100ad0 msgr2=0x7f875806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8758100ad0 0x7f875806ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 -- 172.21.15.45:0/299676764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875819fb20 con 0x7f8758101cd0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f874ffff640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758101cd0 0x7f875819cc10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f874000d950 tx=0x7f874000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f87557fa640 1 -- 172.21.15.45:0/299676764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740017030 con 0x7f8758101cd0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875819fe10 con 0x7f8758101cd0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f87557fa640 1 -- 172.21.15.45:0/299676764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8740004020 con 0x7f8758101cd0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f87557fa640 1 -- 172.21.15.45:0/299676764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87400047a0 con 0x7f8758101cd0 2024-09-10T09:58:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.807+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581a02c0 con 0x7f8758101cd0 2024-09-10T09:58:16.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.809+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8758107f20 con 0x7f8758101cd0 2024-09-10T09:58:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.810+0000 7f87557fa640 1 -- 172.21.15.45:0/299676764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f87400041d0 con 0x7f8758101cd0 2024-09-10T09:58:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.811+0000 7f87557fa640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87440a2b80 0x7f87440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.811+0000 7f87577fe640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87440a2b80 0x7f87440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.811+0000 7f87557fa640 1 -- 172.21.15.45:0/299676764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7f87400cdac0 con 0x7f8758101cd0 2024-09-10T09:58:16.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.811+0000 7f87577fe640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87440a2b80 0x7f87440a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8748005ab0 tx=0x7f8748000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:16.815+0000 7f87557fa640 1 -- 172.21.15.45:0/299676764 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f874009b960 con 0x7f8758101cd0 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:17.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:17.000+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8758062630 con 0x7f87440a2b80 2024-09-10T09:58:17.649 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T09:58:17.275+0000 7fd856f10640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T09:58:17.650 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T09:58:17.287+0000 7fd84f542640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi162 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi162 to 8544M 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:17 smithi162 ceph-mon[34963]: from='osd.5 ' entity='osd.5' 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi162 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi162 to 8544M 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:58:17.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:58:17.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:17.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:17 smithi045 ceph-mon[31114]: from='osd.5 ' entity='osd.5' 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi162 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi162 to 8544M 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:17 smithi047 ceph-mon[35244]: from='osd.5 ' entity='osd.5' 2024-09-10T09:58:18.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:18 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:58:18.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:18 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:58:18.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:18 smithi045 ceph-mon[31114]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:18.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:18 smithi045 ceph-mon[31114]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:58:18.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:18 smithi045 ceph-mon[31114]: osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] boot 2024-09-10T09:58:18.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:18 smithi045 ceph-mon[31114]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:58:18.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:18.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:18 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:58:18.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:18 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:58:18.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:18 smithi162 ceph-mon[34963]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:18.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:18 smithi162 ceph-mon[34963]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:58:18.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:18 smithi162 ceph-mon[34963]: osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] boot 2024-09-10T09:58:18.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:18 smithi162 ceph-mon[34963]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:58:18.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:18.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:18 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:58:18.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:18 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:58:18.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:18 smithi047 ceph-mon[35244]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:58:18.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:18 smithi047 ceph-mon[35244]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:58:18.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:18 smithi047 ceph-mon[35244]: osd.5 [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] boot 2024-09-10T09:58:18.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:18 smithi047 ceph-mon[35244]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:58:18.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:58:19.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:19 smithi045 ceph-mon[31114]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:58:19.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:19 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/834503532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29470075-95e2-48d8-a4f3-88b0b390d2a8"}]: dispatch 2024-09-10T09:58:19.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:19 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/834503532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29470075-95e2-48d8-a4f3-88b0b390d2a8"}]': finished 2024-09-10T09:58:19.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:19 smithi045 ceph-mon[31114]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T09:58:19.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:19.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:19 smithi162 ceph-mon[34963]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:58:19.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:19 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/834503532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29470075-95e2-48d8-a4f3-88b0b390d2a8"}]: dispatch 2024-09-10T09:58:19.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:19 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/834503532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29470075-95e2-48d8-a4f3-88b0b390d2a8"}]': finished 2024-09-10T09:58:19.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:19 smithi162 ceph-mon[34963]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T09:58:19.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:19.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:19 smithi047 ceph-mon[35244]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:58:19.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:19 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/834503532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29470075-95e2-48d8-a4f3-88b0b390d2a8"}]: dispatch 2024-09-10T09:58:19.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:19 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/834503532' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29470075-95e2-48d8-a4f3-88b0b390d2a8"}]': finished 2024-09-10T09:58:19.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:19 smithi047 ceph-mon[35244]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T09:58:19.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:20.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:20 smithi162 ceph-mon[34963]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 568 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:20.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:20 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/4214479957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:58:20.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:20 smithi162 ceph-mon[34963]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T09:58:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:20.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:20 smithi045 ceph-mon[31114]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 568 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:20.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:20 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/4214479957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:58:20.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:20 smithi045 ceph-mon[31114]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T09:58:20.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:20.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:20 smithi047 ceph-mon[35244]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 568 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:20 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/4214479957' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:58:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:20 smithi047 ceph-mon[35244]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T09:58:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:22 smithi162 ceph-mon[34963]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 568 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:22 smithi162 ceph-mon[34963]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T09:58:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:22 smithi045 ceph-mon[31114]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 568 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:22 smithi045 ceph-mon[31114]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T09:58:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:22.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:22 smithi047 ceph-mon[35244]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 568 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:22.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:22 smithi047 ceph-mon[35244]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T09:58:22.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:24 smithi045 ceph-mon[31114]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:24.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:24 smithi162 ceph-mon[34963]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:24.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:24 smithi047 ceph-mon[35244]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:26.560 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:26 smithi045 ceph-mon[31114]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:58:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:26 smithi162 ceph-mon[34963]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:58:26.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:26 smithi047 ceph-mon[35244]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:58:28.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:28 smithi045 ceph-mon[31114]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T09:58:28.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:58:28.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:28 smithi162 ceph-mon[34963]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T09:58:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:58:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:28 smithi047 ceph-mon[35244]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T09:58:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:58:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:29.336 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:29 smithi045 ceph-mon[31114]: Deploying daemon osd.6 on smithi045 2024-09-10T09:58:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:29 smithi162 ceph-mon[34963]: Deploying daemon osd.6 on smithi045 2024-09-10T09:58:29.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:29 smithi047 ceph-mon[35244]: Deploying daemon osd.6 on smithi045 2024-09-10T09:58:30.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:30 smithi045 ceph-mon[31114]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T09:58:30.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:30 smithi162 ceph-mon[34963]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T09:58:30.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:30 smithi047 ceph-mon[35244]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T09:58:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:32 smithi045 ceph-mon[31114]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:58:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:32 smithi162 ceph-mon[34963]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:58:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:32 smithi047 ceph-mon[35244]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:58:34.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:34 smithi045 ceph-mon[31114]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T09:58:34.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:34 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:34.562 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:34 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:34.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:34 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:34.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:34 smithi162 ceph-mon[34963]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T09:58:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:34 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:34 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:34 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:34.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:34 smithi047 ceph-mon[35244]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T09:58:34.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:34 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:34.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:34 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:34.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:34 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:36.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:36 smithi045 ceph-mon[31114]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:58:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:36 smithi162 ceph-mon[34963]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:58:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:36 smithi047 ceph-mon[35244]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:58:37.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.426+0000 7f87557fa640 1 -- 172.21.15.45:0/299676764 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8758062630 con 0x7f87440a2b80 2024-09-10T09:58:37.428 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87440a2b80 msgr2=0x7f87440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87440a2b80 0x7f87440a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8748005ab0 tx=0x7f8748000bc0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758101cd0 msgr2=0x7f875819cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758101cd0 0x7f875819cc10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f874000d950 tx=0x7f874000de10 comp rx=0 tx=0).stop 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 shutdown_connections 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87440a2b80 0x7f87440a5040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8758102690 0x7f875819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8758101cd0 0x7f875819cc10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 --2- 172.21.15.45:0/299676764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8758100ad0 0x7f875806ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.428+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 >> 172.21.15.45:0/299676764 conn(0x7f87580fc280 msgr2=0x7f87581050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.429+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 shutdown_connections 2024-09-10T09:58:37.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:58:37.429+0000 7f875de6c640 1 -- 172.21.15.45:0/299676764 wait complete. 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:37.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:37.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:37.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:37.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:38.058 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T09:58:38.061 INFO:tasks.cephadm:Deploying osd.7 on smithi047 with /dev/vg_nvme/lv_2... 2024-09-10T09:58:38.061 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:58:38.325 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:58:38.599 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:38 smithi045 ceph-mon[31114]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:38.749 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:38 smithi047 ceph-mon[35244]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:38.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:38 smithi162 ceph-mon[34963]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:40.400 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 09:58:40 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T09:58:40.097+0000 7f4c59446640 -1 osd.6 0 log_to_monitors true 2024-09-10T09:58:40.652 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:40 smithi047 ceph-mon[35244]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:40.652 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:40 smithi047 ceph-mon[35244]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:58:40.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:40 smithi045 ceph-mon[31114]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:40.844 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:40 smithi045 ceph-mon[31114]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:58:40.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:40 smithi162 ceph-mon[34963]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:40.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:40 smithi162 ceph-mon[34963]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:58:40.972 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:58:41.003 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-09-10T09:58:41.257 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:58:41.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:41.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:58:41.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:58:41.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:41 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:41.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 -- 172.21.15.47:0/2560309651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 msgr2=0x7f19ec103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/2560309651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec103280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009900 tx=0x7f19d802a410 comp rx=0 tx=0).stop 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 -- 172.21.15.47:0/2560309651 shutdown_connections 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/2560309651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ec1037c0 0x7f19ec10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/2560309651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec103280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/2560309651 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19ec101c00 0x7f19ec102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 -- 172.21.15.47:0/2560309651 >> 172.21.15.47:0/2560309651 conn(0x7f19ec0fd390 msgr2=0x7f19ec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 -- 172.21.15.47:0/2560309651 shutdown_connections 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 -- 172.21.15.47:0/2560309651 wait complete. 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.849+0000 7f19f2dcb640 1 Processor -- start 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f2dcb640 1 -- start start 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f2dcb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ec101c00 0x7f19ec197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f2dcb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:41.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f2dcb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f2dcb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10d900 con 0x7f19ec101c00 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f2dcb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10d780 con 0x7f19ec102e00 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f2dcb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10da80 con 0x7f19ec1037c0 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec1984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35496/0 (socket says 172.21.15.47:35496) 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 -- 172.21.15.47:0/481559891 learned_addr learned my addr 172.21.15.47:0/481559891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f0b67640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ec101c00 0x7f19ec197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19f1368640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 -- 172.21.15.47:0/481559891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19ec1037c0 msgr2=0x7f19ec19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 -- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ec101c00 msgr2=0x7f19ec197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ec101c00 0x7f19ec197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 -- 172.21.15.47:0/481559891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec19fbb0 con 0x7f19ec102e00 2024-09-10T09:58:41.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19ebfff640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec1984c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009230 tx=0x7f19d8004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.850+0000 7f19e9ffb640 1 -- 172.21.15.47:0/481559891 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d803f030 con 0x7f19ec102e00 2024-09-10T09:58:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.851+0000 7f19e9ffb640 1 -- 172.21.15.47:0/481559891 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19d8003ce0 con 0x7f19ec102e00 2024-09-10T09:58:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.851+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec19fe40 con 0x7f19ec102e00 2024-09-10T09:58:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.851+0000 7f19e9ffb640 1 -- 172.21.15.47:0/481559891 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8035070 con 0x7f19ec102e00 2024-09-10T09:58:41.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.851+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec1a0310 con 0x7f19ec102e00 2024-09-10T09:58:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.852+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec108fa0 con 0x7f19ec102e00 2024-09-10T09:58:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.853+0000 7f19e9ffb640 1 -- 172.21.15.47:0/481559891 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f19d8042030 con 0x7f19ec102e00 2024-09-10T09:58:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.853+0000 7f19e9ffb640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19dc0a2bb0 0x7f19dc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.854+0000 7f19e9ffb640 1 -- 172.21.15.47:0/481559891 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7f19d80ee810 con 0x7f19ec102e00 2024-09-10T09:58:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.854+0000 7f19f0b67640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19dc0a2bb0 0x7f19dc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:41.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.854+0000 7f19f0b67640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19dc0a2bb0 0x7f19dc0a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f19ec102c60 tx=0x7f19e0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:41.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:41.858+0000 7f19e9ffb640 1 -- 172.21.15.47:0/481559891 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19d80bc5a0 con 0x7f19ec102e00 2024-09-10T09:58:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:58:41.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:58:41.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:41 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:41.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T09:58:41.463+0000 7f4c54ba6640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T09:58:41.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T09:58:41.476+0000 7f4c511e0640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:58:41.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:58:41.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:58:41.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:58:41.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:41 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:42.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:58:42.040+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f19ec062630 con 0x7f19dc0a2bb0 2024-09-10T09:58:42.742 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:42.742 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi045 2024-09-10T09:58:42.742 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi045 to 5697M 2024-09-10T09:58:42.742 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:42.742 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:58:42.742 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:58:42.742 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:42.743 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi045 2024-09-10T09:58:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi045 to 5697M 2024-09-10T09:58:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:58:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:58:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi045 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi045 to 5697M 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:42.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:43.765 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:43 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:58:43.765 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:43 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:58:43.765 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:43 smithi047 ceph-mon[35244]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:58:43.765 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:43 smithi047 ceph-mon[35244]: osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] boot 2024-09-10T09:58:43.765 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:43 smithi047 ceph-mon[35244]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T09:58:43.765 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:43 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:43 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:58:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:43 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:43 smithi162 ceph-mon[34963]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:43 smithi162 ceph-mon[34963]: osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] boot 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:43 smithi162 ceph-mon[34963]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:43 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:43 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:43 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:43 smithi045 ceph-mon[31114]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:58:43.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:43 smithi045 ceph-mon[31114]: osd.6 [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] boot 2024-09-10T09:58:43.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:43 smithi045 ceph-mon[31114]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T09:58:43.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:43 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:58:44.780 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:44 smithi047 ceph-mon[35244]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:44.781 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:44 smithi047 ceph-mon[35244]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:58:44.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:44 smithi045 ceph-mon[31114]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:44.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:44 smithi045 ceph-mon[31114]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:58:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:44 smithi162 ceph-mon[34963]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:58:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:44 smithi162 ceph-mon[34963]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:58:45.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:45 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/1695833860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]: dispatch 2024-09-10T09:58:45.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:45 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]: dispatch 2024-09-10T09:58:45.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:45 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]': finished 2024-09-10T09:58:45.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:45 smithi162 ceph-mon[34963]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T09:58:45.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:58:45.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:45 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/1774677747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:58:45.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:45 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/1695833860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]: dispatch 2024-09-10T09:58:45.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:45 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]: dispatch 2024-09-10T09:58:45.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:45 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]': finished 2024-09-10T09:58:45.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:45 smithi045 ceph-mon[31114]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T09:58:45.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:58:45.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:45 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/1774677747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:58:45.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:45 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/1695833860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]: dispatch 2024-09-10T09:58:45.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:45 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]: dispatch 2024-09-10T09:58:45.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:45 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32004f02-5b68-43bf-9af9-f9cabebe8043"}]': finished 2024-09-10T09:58:45.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:45 smithi047 ceph-mon[35244]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T09:58:45.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:58:45.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:45 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/1774677747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:58:46.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:46 smithi162 ceph-mon[34963]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:46.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:46 smithi045 ceph-mon[31114]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:46.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:46 smithi047 ceph-mon[35244]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:48.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:48 smithi162 ceph-mon[34963]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:48.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:48 smithi045 ceph-mon[31114]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:48.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:48 smithi047 ceph-mon[35244]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:50.790 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:50 smithi047 ceph-mon[35244]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:50 smithi162 ceph-mon[34963]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:50.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:50 smithi045 ceph-mon[31114]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:52.774 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:52 smithi047 ceph-mon[35244]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:52.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:52 smithi162 ceph-mon[34963]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:52.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:52 smithi045 ceph-mon[31114]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:58:53.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:53.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:58:53.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:53.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:58:53.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:54.737 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:54 smithi047 ceph-mon[35244]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:54.737 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:54 smithi047 ceph-mon[35244]: Deploying daemon osd.7 on smithi047 2024-09-10T09:58:54.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:54 smithi162 ceph-mon[34963]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:54.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:54 smithi162 ceph-mon[34963]: Deploying daemon osd.7 on smithi047 2024-09-10T09:58:54.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:54 smithi045 ceph-mon[31114]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:54.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:54 smithi045 ceph-mon[31114]: Deploying daemon osd.7 on smithi047 2024-09-10T09:58:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:56 smithi047 ceph-mon[35244]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:56.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:56 smithi162 ceph-mon[34963]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:56.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:56 smithi045 ceph-mon[31114]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:57.778 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:57 smithi047 ceph-mon[35244]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:57 smithi162 ceph-mon[34963]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:57.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:57 smithi045 ceph-mon[31114]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:59 smithi047 ceph-mon[35244]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:59.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:59 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:59.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:59 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:59.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:58:59 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:59.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:59 smithi162 ceph-mon[34963]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:59.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:59 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:59.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:59 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:59.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:58:59 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:59.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:59 smithi045 ceph-mon[31114]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:58:59.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:59 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:59.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:59 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:58:59.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:58:59 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.236+0000 7f19e9ffb640 1 -- 172.21.15.47:0/481559891 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f19ec062630 con 0x7f19dc0a2bb0 2024-09-10T09:59:02.237 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 7 on host 'smithi047' 2024-09-10T09:59:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.238+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19dc0a2bb0 msgr2=0x7f19dc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19dc0a2bb0 0x7f19dc0a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f19ec102c60 tx=0x7f19e0009110 comp rx=0 tx=0).stop 2024-09-10T09:59:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 msgr2=0x7f19ec1984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec1984c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009230 tx=0x7f19d8004360 comp rx=0 tx=0).stop 2024-09-10T09:59:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 shutdown_connections 2024-09-10T09:59:02.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19dc0a2bb0 0x7f19dc0a5070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f19ec102e00 0x7f19ec1984c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 --2- 172.21.15.47:0/481559891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19ec101c00 0x7f19ec197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 >> 172.21.15.47:0/481559891 conn(0x7f19ec0fd390 msgr2=0x7f19ec106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 shutdown_connections 2024-09-10T09:59:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:02.239+0000 7f19f2dcb640 1 -- 172.21.15.47:0/481559891 wait complete. 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:02.559 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:02 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:02 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:02.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:02.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:02.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:02.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:02 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:02.861 DEBUG:teuthology.orchestra.run.smithi047:osd.7> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T09:59:02.864 INFO:tasks.cephadm:Deploying osd.8 on smithi162 with /dev/vg_nvme/lv_2... 2024-09-10T09:59:02.864 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:59:03.117 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:59:03.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:03 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:03.535 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:03 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:03.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:03 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:04.345 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:04 smithi047 ceph-mon[35244]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:04.510 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:04 smithi162 ceph-mon[34963]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:04.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:04 smithi045 ceph-mon[31114]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:05.250 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 09:59:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T09:59:04.966+0000 7f33cedf6640 -1 osd.7 0 log_to_monitors true 2024-09-10T09:59:05.604 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:05 smithi162 ceph-mon[34963]: from='osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:59:05.605 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:05 smithi162 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:59:05.607 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T09:59:05.633 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_2 2024-09-10T09:59:05.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:05 smithi045 ceph-mon[31114]: from='osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:59:05.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:05 smithi045 ceph-mon[31114]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:59:05.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:05 smithi047 ceph-mon[35244]: from='osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:59:05.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:05 smithi047 ceph-mon[35244]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:59:05.888 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:59:06.278 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:06.278 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:59:06.278 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:59:06.278 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:59:06.279 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:06.279 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:59:06.279 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.279 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.279 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.279 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.280 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.280 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.280 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:06.280 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:06.280 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:06 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.325 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 -- 172.21.15.162:0/2994927171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754102690 msgr2=0x7fe754108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.325 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 --2- 172.21.15.162:0/2994927171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754102690 0x7fe754108f20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009ca0 tx=0x7fe74c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.325 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 -- 172.21.15.162:0/2994927171 shutdown_connections 2024-09-10T09:59:06.325 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 --2- 172.21.15.162:0/2994927171 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754102690 0x7fe754108f20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.325 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 --2- 172.21.15.162:0/2994927171 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe754101cd0 0x7fe754102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.325 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 --2- 172.21.15.162:0/2994927171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe754100ad0 0x7fe754100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 -- 172.21.15.162:0/2994927171 >> 172.21.15.162:0/2994927171 conn(0x7fe7540fc280 msgr2=0x7fe7540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:06.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 -- 172.21.15.162:0/2994927171 shutdown_connections 2024-09-10T09:59:06.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.302+0000 7fe75a551640 1 -- 172.21.15.162:0/2994927171 wait complete. 2024-09-10T09:59:06.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 Processor -- start 2024-09-10T09:59:06.326 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 -- start start 2024-09-10T09:59:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe754100ad0 0x7fe754197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754101cd0 0x7fe754198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe754102690 0x7fe75419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe75410c750 con 0x7fe754102690 2024-09-10T09:59:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe75410c5d0 con 0x7fe754100ad0 2024-09-10T09:59:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe75a551640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe75410c8d0 con 0x7fe754101cd0 2024-09-10T09:59:06.327 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe7537fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754101cd0 0x7fe754198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.328 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe7537fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754101cd0 0x7fe754198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51842/0 (socket says 172.21.15.162:51842) 2024-09-10T09:59:06.328 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe7537fe640 1 -- 172.21.15.162:0/3564897041 learned_addr learned my addr 172.21.15.162:0/3564897041 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T09:59:06.328 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.303+0000 7fe753fff640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe754100ad0 0x7fe754197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.328 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe758aee640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe754102690 0x7fe75419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.328 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7537fe640 1 -- 172.21.15.162:0/3564897041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe754100ad0 msgr2=0x7fe754197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7537fe640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe754100ad0 0x7fe754197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7537fe640 1 -- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe754102690 msgr2=0x7fe75419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:06.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7537fe640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe754102690 0x7fe75419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:06.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7537fe640 1 -- 172.21.15.162:0/3564897041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75419fa80 con 0x7fe754101cd0 2024-09-10T09:59:06.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe758aee640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe754102690 0x7fe75419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:06.329 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7537fe640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754101cd0 0x7fe754198400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e9f0 tx=0x7fe74800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:06.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7517fa640 1 -- 172.21.15.162:0/3564897041 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74800cd80 con 0x7fe754101cd0 2024-09-10T09:59:06.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75419fd70 con 0x7fe754101cd0 2024-09-10T09:59:06.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7517fa640 1 -- 172.21.15.162:0/3564897041 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe748005500 con 0x7fe754101cd0 2024-09-10T09:59:06.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe7517fa640 1 -- 172.21.15.162:0/3564897041 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748010690 con 0x7fe754101cd0 2024-09-10T09:59:06.330 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.304+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7541a0200 con 0x7fe754101cd0 2024-09-10T09:59:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.305+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7541a0490 con 0x7fe754101cd0 2024-09-10T09:59:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.306+0000 7fe7517fa640 1 -- 172.21.15.162:0/3564897041 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe7480046c0 con 0x7fe754101cd0 2024-09-10T09:59:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.307+0000 7fe7517fa640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe73c0a2bb0 0x7fe73c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.307+0000 7fe7517fa640 1 -- 172.21.15.162:0/3564897041 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7fe7480cea20 con 0x7fe754101cd0 2024-09-10T09:59:06.331 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.307+0000 7fe753fff640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe73c0a2bb0 0x7fe73c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:06.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.308+0000 7fe753fff640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe73c0a2bb0 0x7fe73c0a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe754101b30 tx=0x7fe744005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:06.332 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.310+0000 7fe7517fa640 1 -- 172.21.15.162:0/3564897041 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7480541b0 con 0x7fe754101cd0 2024-09-10T09:59:06.489 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:06.488+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe7541a0660 con 0x7fe73c0a2bb0 2024-09-10T09:59:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:06.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:59:06.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:59:06.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:59:06.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:06.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:59:06.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:06.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:06.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:06 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:06.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:59:06.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:59:06.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:59:06.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:06.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T09:59:06.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:06.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:06.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:06.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:06.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:07.258 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T09:59:06.948+0000 7f33ca556640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T09:59:07.258 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 09:59:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T09:59:06.959+0000 7f33c2b88640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi047 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi047 to 5697M 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:07.572 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:07 smithi162 ceph-mon[34963]: from='osd.7 ' entity='osd.7' 2024-09-10T09:59:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi047 2024-09-10T09:59:07.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi047 to 5697M 2024-09-10T09:59:07.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:59:07.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:59:07.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:07.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:07.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:07.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:07.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:07.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:07.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:07 smithi045 ceph-mon[31114]: from='osd.7 ' entity='osd.7' 2024-09-10T09:59:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi047 2024-09-10T09:59:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi047 to 5697M 2024-09-10T09:59:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T09:59:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:59:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:07 smithi047 ceph-mon[35244]: from='osd.7 ' entity='osd.7' 2024-09-10T09:59:08.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:08 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:59:08.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:08 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:59:08.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:08 smithi162 ceph-mon[34963]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:08.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:08.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:08 smithi162 ceph-mon[34963]: osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] boot 2024-09-10T09:59:08.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:08 smithi162 ceph-mon[34963]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T09:59:08.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:08 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:08 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:59:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:08 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:59:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:08 smithi045 ceph-mon[31114]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:08 smithi045 ceph-mon[31114]: osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] boot 2024-09-10T09:59:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:08 smithi045 ceph-mon[31114]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T09:59:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:08 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:08.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:08 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:59:08.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:08 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:59:08.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:08 smithi047 ceph-mon[35244]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:59:08.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:08.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:08 smithi047 ceph-mon[35244]: osd.7 [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] boot 2024-09-10T09:59:08.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:08 smithi047 ceph-mon[35244]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T09:59:08.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:08 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:59:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:09 smithi162 ceph-mon[34963]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:59:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:09 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/3229375014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]: dispatch 2024-09-10T09:59:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:09 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]: dispatch 2024-09-10T09:59:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:09 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]': finished 2024-09-10T09:59:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:09 smithi162 ceph-mon[34963]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T09:59:09.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:09 smithi045 ceph-mon[31114]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:59:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:09 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/3229375014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]: dispatch 2024-09-10T09:59:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:09 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]: dispatch 2024-09-10T09:59:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:09 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]': finished 2024-09-10T09:59:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:09 smithi045 ceph-mon[31114]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T09:59:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:09 smithi047 ceph-mon[35244]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:59:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:09 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/3229375014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]: dispatch 2024-09-10T09:59:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:09 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]: dispatch 2024-09-10T09:59:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:09 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3e8cc6aa-174a-4685-a735-e8a9aed57f04"}]': finished 2024-09-10T09:59:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:09 smithi047 ceph-mon[35244]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T09:59:09.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:10.523 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:10 smithi047 ceph-mon[35244]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:10.524 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:10 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/648241173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:10.524 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:10 smithi047 ceph-mon[35244]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T09:59:10.524 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:10 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:10 smithi162 ceph-mon[34963]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:10 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/648241173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:10 smithi162 ceph-mon[34963]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T09:59:10.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:10 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:10.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:10 smithi045 ceph-mon[31114]: pgmap v170: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:10.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:10 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/648241173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:10.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:10 smithi045 ceph-mon[31114]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T09:59:10.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:10 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:11 smithi162 ceph-mon[34963]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:11 smithi162 ceph-mon[34963]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T09:59:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:11 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:12.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:11 smithi045 ceph-mon[31114]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:12.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:11 smithi045 ceph-mon[31114]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T09:59:12.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:11 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:12.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:11 smithi047 ceph-mon[35244]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:12.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:11 smithi047 ceph-mon[35244]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T09:59:12.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:11 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:14.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:13 smithi162 ceph-mon[34963]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:13 smithi045 ceph-mon[31114]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:14.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:13 smithi047 ceph-mon[35244]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:15 smithi162 ceph-mon[34963]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:15 smithi045 ceph-mon[31114]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:16.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:15 smithi047 ceph-mon[35244]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:18.145 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:17 smithi162 ceph-mon[34963]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:18.145 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:59:18.145 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:18.145 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:17 smithi162 ceph-mon[34963]: Deploying daemon osd.8 on smithi162 2024-09-10T09:59:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:17 smithi045 ceph-mon[31114]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:59:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:17 smithi045 ceph-mon[31114]: Deploying daemon osd.8 on smithi162 2024-09-10T09:59:18.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:17 smithi047 ceph-mon[35244]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:18.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:59:18.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:18.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:17 smithi047 ceph-mon[35244]: Deploying daemon osd.8 on smithi162 2024-09-10T09:59:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:19 smithi162 ceph-mon[34963]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:20.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:19 smithi045 ceph-mon[31114]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:20.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:19 smithi047 ceph-mon[35244]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:21.996 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:21 smithi162 ceph-mon[34963]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:22.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:21 smithi045 ceph-mon[31114]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:22.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:21 smithi047 ceph-mon[35244]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:23.927 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:23 smithi162 ceph-mon[34963]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:23.927 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:23 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:23.927 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:23 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:23.927 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:23 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:24.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:23 smithi045 ceph-mon[31114]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:24.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:23 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:24.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:23 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:24.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:23 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:24.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:23 smithi047 ceph-mon[35244]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:24.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:23 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:24.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:23 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:24.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:23 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:26.106 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:25 smithi162 ceph-mon[34963]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:26.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:25 smithi045 ceph-mon[31114]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:26.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:25 smithi047 ceph-mon[35244]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:26.443 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.442+0000 7fe7517fa640 1 -- 172.21.15.162:0/3564897041 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe7541a0660 con 0x7fe73c0a2bb0 2024-09-10T09:59:26.444 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 8 on host 'smithi162' 2024-09-10T09:59:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe73c0a2bb0 msgr2=0x7fe73c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe73c0a2bb0 0x7fe73c0a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe754101b30 tx=0x7fe744005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754101cd0 msgr2=0x7fe754198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754101cd0 0x7fe754198400 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe74800e9f0 tx=0x7fe74800edf0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 shutdown_connections 2024-09-10T09:59:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe73c0a2bb0 0x7fe73c0a5070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.446 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe754102690 0x7fe75419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe754101cd0 0x7fe754198400 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 --2- 172.21.15.162:0/3564897041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe754100ad0 0x7fe754197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 >> 172.21.15.162:0/3564897041 conn(0x7fe7540fc280 msgr2=0x7fe7540fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 shutdown_connections 2024-09-10T09:59:26.447 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T09:59:26.445+0000 7fe75a551640 1 -- 172.21.15.162:0/3564897041 wait complete. 2024-09-10T09:59:27.044 DEBUG:teuthology.orchestra.run.smithi162:osd.8> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T09:59:27.047 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2024-09-10T09:59:27.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:59:27.317 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:27 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.719 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.719 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.719 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.719 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:27 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:27.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:27 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:28.586 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:28 smithi162 ceph-mon[34963]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:28.728 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:28 smithi045 ceph-mon[31114]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:28.811 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:28 smithi047 ceph-mon[35244]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:29.765 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 09:59:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T09:59:29.514+0000 7f0c4e285640 -1 osd.8 0 log_to_monitors true 2024-09-10T09:59:29.901 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T09:59:29.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-09-10T09:59:30.188 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:30.568 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:30.569 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:30.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:30.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:30.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.653+0000 7f645113b640 1 -- 172.21.15.45:0/2019279238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c107900 msgr2=0x7f644c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.653+0000 7f645113b640 1 --2- 172.21.15.45:0/2019279238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c107900 0x7f644c109d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6444009ca0 tx=0x7f644402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.653+0000 7f645113b640 1 -- 172.21.15.45:0/2019279238 shutdown_connections 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.653+0000 7f645113b640 1 --2- 172.21.15.45:0/2019279238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c107900 0x7f644c109d50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.654+0000 7f645113b640 1 --2- 172.21.15.45:0/2019279238 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f644c104fd0 0x7f644c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.654+0000 7f645113b640 1 --2- 172.21.15.45:0/2019279238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c1026a0 0x7f644c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.654+0000 7f645113b640 1 -- 172.21.15.45:0/2019279238 >> 172.21.15.45:0/2019279238 conn(0x7f644c0fc2a0 msgr2=0x7f644c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.654+0000 7f645113b640 1 -- 172.21.15.45:0/2019279238 shutdown_connections 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.654+0000 7f645113b640 1 -- 172.21.15.45:0/2019279238 wait complete. 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.654+0000 7f645113b640 1 Processor -- start 2024-09-10T09:59:30.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f645113b640 1 -- start start 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f645113b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c1026a0 0x7f644c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f645113b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c104fd0 0x7f644c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f645113b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f644c107900 0x7f644c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f645113b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f644c10d8a0 con 0x7f644c104fd0 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f645113b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f644c10d720 con 0x7f644c1026a0 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f645113b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f644c10da20 con 0x7f644c107900 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c104fd0 0x7f644c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c104fd0 0x7f644c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41778/0 (socket says 172.21.15.45:41778) 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 -- 172.21.15.45:0/3548194190 learned_addr learned my addr 172.21.15.45:0/3548194190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644b59e640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f644c107900 0x7f644c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644ad9d640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c1026a0 0x7f644c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 -- 172.21.15.45:0/3548194190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f644c107900 msgr2=0x7f644c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f644c107900 0x7f644c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 -- 172.21.15.45:0/3548194190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c1026a0 msgr2=0x7f644c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c1026a0 0x7f644c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.655+0000 7f644a59c640 1 -- 172.21.15.45:0/3548194190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c19fa70 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.656+0000 7f644a59c640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c104fd0 0x7f644c198380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f643400d770 tx=0x7f643400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.656+0000 7f6443fff640 1 -- 172.21.15.45:0/3548194190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434017070 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.656+0000 7f6443fff640 1 -- 172.21.15.45:0/3548194190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6434010460 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.656+0000 7f6443fff640 1 -- 172.21.15.45:0/3548194190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643400f620 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.656+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c19fd60 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.656+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c1a0240 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.657+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c0fdd90 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.658+0000 7f6443fff640 1 -- 172.21.15.45:0/3548194190 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f643401a070 con 0x7f644c104fd0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.659+0000 7f6443fff640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64380a2bb0 0x7f64380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.659+0000 7f644ad9d640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64380a2bb0 0x7f64380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:30.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.659+0000 7f6443fff640 1 -- 172.21.15.45:0/3548194190 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5533+0+0 (secure 0 0 0) 0x7f64340cdf00 con 0x7f644c104fd0 2024-09-10T09:59:30.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.660+0000 7f644ad9d640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64380a2bb0 0x7f64380a5070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f643c00abe0 tx=0x7f643c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:30.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.662+0000 7f6443fff640 1 -- 172.21.15.45:0/3548194190 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f643409b940 con 0x7f644c104fd0 2024-09-10T09:59:30.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:30.846+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f644c062630 con 0x7f64380a2bb0 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:30.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:30.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:30.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:30.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:30.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:30.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi162 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi162 to 5696M 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: from='osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:59:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:31.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:31.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:31.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T09:59:31.506+0000 7f0c499e5640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T09:59:31.651 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 09:59:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T09:59:31.517+0000 7f0c4601f640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:59:31.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi162 2024-09-10T09:59:31.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi162 to 5696M 2024-09-10T09:59:31.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:59:31.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:59:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: from='osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:59:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:59:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:31.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi162 2024-09-10T09:59:31.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi162 to 5696M 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: from='osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:32 smithi162 ceph-mon[34963]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:32.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:32 smithi162 ceph-mon[34963]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:32.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:32 smithi162 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:59:32.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:32 smithi162 ceph-mon[34963]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:59:32.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:32.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:32.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:32 smithi045 ceph-mon[31114]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:32.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:32 smithi045 ceph-mon[31114]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:32.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:32 smithi045 ceph-mon[31114]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:59:32.875 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:32 smithi045 ceph-mon[31114]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:59:32.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:32.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:32.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:32 smithi047 ceph-mon[35244]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:32.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:32 smithi047 ceph-mon[35244]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:32.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:32 smithi047 ceph-mon[35244]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T09:59:32.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:32 smithi047 ceph-mon[35244]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:59:32.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:32.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:33.765 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:59:33.765 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] boot 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/4023564824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ab7cd4-9ec8-46b4-9a89-5d954e80b692"}]: dispatch 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/4023564824' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ab7cd4-9ec8-46b4-9a89-5d954e80b692"}]': finished 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T09:59:33.766 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:33 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] boot 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/4023564824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ab7cd4-9ec8-46b4-9a89-5d954e80b692"}]: dispatch 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/4023564824' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ab7cd4-9ec8-46b4-9a89-5d954e80b692"}]': finished 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T09:59:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:33 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: osd.8 [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] boot 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/4023564824' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12ab7cd4-9ec8-46b4-9a89-5d954e80b692"}]: dispatch 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/4023564824' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12ab7cd4-9ec8-46b4-9a89-5d954e80b692"}]': finished 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T09:59:33.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:33 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:34 smithi162 ceph-mon[34963]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:34.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:34 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3199878002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:34 smithi162 ceph-mon[34963]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T09:59:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:34 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:34.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:34 smithi045 ceph-mon[31114]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:34.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:34 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3199878002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:34.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:34 smithi045 ceph-mon[31114]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T09:59:34.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:34 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:34.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:34 smithi047 ceph-mon[35244]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:59:34.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:34 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3199878002' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:34.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:34 smithi047 ceph-mon[35244]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T09:59:34.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:34 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:36 smithi162 ceph-mon[34963]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:36.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:36 smithi162 ceph-mon[34963]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T09:59:36.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:36 smithi045 ceph-mon[31114]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:36 smithi045 ceph-mon[31114]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T09:59:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:36 smithi047 ceph-mon[35244]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:36 smithi047 ceph-mon[35244]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T09:59:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:37 smithi162 ceph-mon[34963]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T09:59:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:37 smithi045 ceph-mon[31114]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T09:59:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:37 smithi047 ceph-mon[35244]: osdmap e60: 10 total, 9 up, 10 in 2024-09-10T09:59:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:38.348 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:38 smithi045 ceph-mon[31114]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:38.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:38 smithi162 ceph-mon[34963]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:38.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:38 smithi047 ceph-mon[35244]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:40.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:40 smithi045 ceph-mon[31114]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-10T09:59:40.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:40 smithi162 ceph-mon[34963]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-10T09:59:40.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:40 smithi047 ceph-mon[35244]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-10T09:59:42.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:42 smithi045 ceph-mon[31114]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T09:59:42.536 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:59:42.536 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:42.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:42 smithi162 ceph-mon[34963]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T09:59:42.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:59:42.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:42.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:42 smithi047 ceph-mon[35244]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T09:59:42.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:59:42.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:43.485 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:43 smithi045 ceph-mon[31114]: Deploying daemon osd.9 on smithi045 2024-09-10T09:59:43.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:43 smithi162 ceph-mon[34963]: Deploying daemon osd.9 on smithi045 2024-09-10T09:59:43.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:43 smithi047 ceph-mon[35244]: Deploying daemon osd.9 on smithi045 2024-09-10T09:59:44.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:44 smithi045 ceph-mon[31114]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:59:44.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:44 smithi162 ceph-mon[34963]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:59:44.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:44 smithi047 ceph-mon[35244]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:59:46.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:46 smithi045 ceph-mon[31114]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T09:59:46.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:46 smithi162 ceph-mon[34963]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T09:59:46.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:46 smithi047 ceph-mon[35244]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T09:59:48.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:48 smithi162 ceph-mon[34963]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T09:59:48.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:48.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:48.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:48 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:48 smithi045 ceph-mon[31114]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T09:59:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:48 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:48 smithi047 ceph-mon[35244]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T09:59:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:48.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:48 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:50.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:50 smithi045 ceph-mon[31114]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:59:50.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:50.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:50 smithi162 ceph-mon[34963]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:59:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:50.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:50 smithi047 ceph-mon[35244]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:59:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.020+0000 7f6443fff640 1 -- 172.21.15.45:0/3548194190 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f644c062630 con 0x7f64380a2bb0 2024-09-10T09:59:51.023 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2024-09-10T09:59:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64380a2bb0 msgr2=0x7f64380a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:51.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64380a2bb0 0x7f64380a5070 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f643c00abe0 tx=0x7f643c00a2d0 comp rx=0 tx=0).stop 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c104fd0 msgr2=0x7f644c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c104fd0 0x7f644c198380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f643400d770 tx=0x7f643400db70 comp rx=0 tx=0).stop 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 shutdown_connections 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64380a2bb0 0x7f64380a5070 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f644c107900 0x7f644c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f644c104fd0 0x7f644c198380 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 --2- 172.21.15.45:0/3548194190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f644c1026a0 0x7f644c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 >> 172.21.15.45:0/3548194190 conn(0x7f644c0fc2a0 msgr2=0x7f644c102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 shutdown_connections 2024-09-10T09:59:51.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T09:59:51.023+0000 7f645113b640 1 -- 172.21.15.45:0/3548194190 wait complete. 2024-09-10T09:59:51.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:51.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:51 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:51.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:51 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:51.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:51.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:51.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:51 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:51.769 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T09:59:51.772 INFO:tasks.cephadm:Deploying osd.10 on smithi047 with /dev/vg_nvme/lv_1... 2024-09-10T09:59:51.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:59:52.034 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:59:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:52 smithi045 ceph-mon[31114]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:52.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:52 smithi047 ceph-mon[35244]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:52.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:52 smithi162 ceph-mon[34963]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:54.131 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 09:59:53 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T09:59:53.872+0000 7f96e8294640 -1 osd.9 0 log_to_monitors true 2024-09-10T09:59:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:54 smithi045 ceph-mon[31114]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:54 smithi045 ceph-mon[31114]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:59:54.403 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:54 smithi047 ceph-mon[35244]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:54.403 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:54 smithi047 ceph-mon[35244]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:59:54.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:54 smithi162 ceph-mon[34963]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:54.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:54 smithi162 ceph-mon[34963]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:59:54.674 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-09-10T09:59:54.701 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-09-10T09:59:54.956 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:59:55.341 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:59:55.341 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T09:59:55.341 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.342 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:55 smithi047 ceph-mon[35244]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2946681330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20101cd0 msgr2=0x7f4d20102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2946681330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20101cd0 0x7f4d20102150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009900 tx=0x7f4d0802a410 comp rx=0 tx=0).stop 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2946681330 shutdown_connections 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2946681330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d20102690 0x7f4d20108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2946681330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20101cd0 0x7f4d20102150 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2946681330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d20100ad0 0x7f4d20100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2946681330 >> 172.21.15.47:0/2946681330 conn(0x7f4d200fc280 msgr2=0x7f4d200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2946681330 shutdown_connections 2024-09-10T09:59:55.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.427+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2946681330 wait complete. 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 Processor -- start 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 -- start start 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20100ad0 0x7f4d20197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d20101cd0 0x7f4d20198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d20102690 0x7f4d2019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2010c730 con 0x7f4d20101cd0 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2010c5b0 con 0x7f4d20100ad0 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d24ddc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2010c8b0 con 0x7f4d20102690 2024-09-10T09:59:55.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d1e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20100ad0 0x7f4d20197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d1e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20100ad0 0x7f4d20197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45512/0 (socket says 172.21.15.47:45512) 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d1e59c640 1 -- 172.21.15.47:0/2538893731 learned_addr learned my addr 172.21.15.47:0/2538893731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d1dd9b640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d20101cd0 0x7f4d20198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d1ed9d640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d20102690 0x7f4d2019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.428+0000 7f4d1e59c640 1 -- 172.21.15.47:0/2538893731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d20102690 msgr2=0x7f4d2019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d1e59c640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d20102690 0x7f4d2019f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d1e59c640 1 -- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d20101cd0 msgr2=0x7f4d20198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d1e59c640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d20101cd0 0x7f4d20198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d1e59c640 1 -- 172.21.15.47:0/2538893731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2019fb70 con 0x7f4d20100ad0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d1e59c640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20100ad0 0x7f4d20197f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e9c0 tx=0x7f4d1000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d177fe640 1 -- 172.21.15.47:0/2538893731 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000cd80 con 0x7f4d20100ad0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d177fe640 1 -- 172.21.15.47:0/2538893731 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d100035f0 con 0x7f4d20100ad0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2019fe60 con 0x7f4d20100ad0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d177fe640 1 -- 172.21.15.47:0/2538893731 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10010690 con 0x7f4d20100ad0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.429+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d201a02b0 con 0x7f4d20100ad0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.430+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d201a0540 con 0x7f4d20100ad0 2024-09-10T09:59:55.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.431+0000 7f4d177fe640 1 -- 172.21.15.47:0/2538893731 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4d100037a0 con 0x7f4d20100ad0 2024-09-10T09:59:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.432+0000 7f4d177fe640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d0c0a2bb0 0x7f4d0c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.432+0000 7f4d177fe640 1 -- 172.21.15.47:0/2538893731 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f4d100cea90 con 0x7f4d20100ad0 2024-09-10T09:59:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.432+0000 7f4d1dd9b640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d0c0a2bb0 0x7f4d0c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.433+0000 7f4d1dd9b640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d0c0a2bb0 0x7f4d0c0a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d080314e0 tx=0x7f4d08000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.436+0000 7f4d177fe640 1 -- 172.21.15.47:0/2538893731 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d1009c310 con 0x7f4d20100ad0 2024-09-10T09:59:55.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T09:59:55.620+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4d2006a6f0 con 0x7f4d0c0a2bb0 2024-09-10T09:59:55.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:55.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:55.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:55 smithi162 ceph-mon[34963]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:59:55.651 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T09:59:55.273+0000 7f96e39f4640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T09:59:55.651 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T09:59:55.286+0000 7f96e002e640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:59:55.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: osdmap e61: 10 total, 9 up, 10 in 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T09:59:55.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:55 smithi045 ceph-mon[31114]: from='osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-09-10T09:59:56.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:56.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi045 2024-09-10T09:59:56.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi045 to 4273M 2024-09-10T09:59:56.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:59:56.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:56 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi045 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi045 to 4273M 2024-09-10T09:59:56.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:59:56.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:56.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:56.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:56.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:56 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi045 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi045 to 4273M 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:56.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:56 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:57.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:57 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T09:59:57.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:57 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T09:59:57.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:57 smithi047 ceph-mon[35244]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:57.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:57 smithi047 ceph-mon[35244]: osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] boot 2024-09-10T09:59:57.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:57 smithi047 ceph-mon[35244]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T09:59:57.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:57 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:57.621 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:57 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T09:59:57.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:57 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T09:59:57.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:57 smithi045 ceph-mon[31114]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:57.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:57 smithi045 ceph-mon[31114]: osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] boot 2024-09-10T09:59:57.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:57 smithi045 ceph-mon[31114]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T09:59:57.622 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:57 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:57.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:57 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T09:59:57.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:57 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T09:59:57.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:57 smithi162 ceph-mon[34963]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:59:57.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:57 smithi162 ceph-mon[34963]: osd.9 [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] boot 2024-09-10T09:59:57.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:57 smithi162 ceph-mon[34963]: osdmap e63: 10 total, 10 up, 10 in 2024-09-10T09:59:57.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:57 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:59:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:58 smithi047 ceph-mon[35244]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:58 smithi047 ceph-mon[35244]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T09:59:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:58 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/1035455847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]: dispatch 2024-09-10T09:59:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:58 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]: dispatch 2024-09-10T09:59:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:58 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]': finished 2024-09-10T09:59:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:58 smithi047 ceph-mon[35244]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T09:59:58.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:59:58.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:58 smithi162 ceph-mon[34963]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:58 smithi162 ceph-mon[34963]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:58 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/1035455847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]: dispatch 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:58 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]: dispatch 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:58 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]': finished 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:58 smithi162 ceph-mon[34963]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:58 smithi045 ceph-mon[31114]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:58 smithi045 ceph-mon[31114]: osdmap e64: 10 total, 10 up, 10 in 2024-09-10T09:59:58.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:58 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/1035455847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]: dispatch 2024-09-10T09:59:58.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:58 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]: dispatch 2024-09-10T09:59:58.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:58 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb823514-17df-4818-bfdf-7b8cdbcfbcd4"}]': finished 2024-09-10T09:59:58.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:58 smithi045 ceph-mon[31114]: osdmap e65: 11 total, 10 up, 11 in 2024-09-10T09:59:58.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:59:59.551 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 09:59:59 smithi162 ceph-mon[34963]: from='client.? 172.21.15.47:0/700869398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:59.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 09:59:59 smithi045 ceph-mon[31114]: from='client.? 172.21.15.47:0/700869398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:59:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 09:59:59 smithi047 ceph-mon[35244]: from='client.? 172.21.15.47:0/700869398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:00:00.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:00 smithi047 ceph-mon[35244]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:00.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:00 smithi047 ceph-mon[35244]: overall HEALTH_OK 2024-09-10T10:00:00.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:00 smithi162 ceph-mon[34963]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:00.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:00 smithi162 ceph-mon[34963]: overall HEALTH_OK 2024-09-10T10:00:00.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:00 smithi045 ceph-mon[31114]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:00.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:00 smithi045 ceph-mon[31114]: overall HEALTH_OK 2024-09-10T10:00:02.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:02 smithi047 ceph-mon[35244]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:02.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:02 smithi162 ceph-mon[34963]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:02.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:02 smithi045 ceph-mon[31114]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:04.331 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:04 smithi047 ceph-mon[35244]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:04 smithi162 ceph-mon[34963]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:04.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:04 smithi045 ceph-mon[31114]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:06.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:06 smithi162 ceph-mon[34963]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:06 smithi045 ceph-mon[31114]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:06.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:06 smithi047 ceph-mon[35244]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:07.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:00:07.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:07.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:07 smithi047 ceph-mon[35244]: Deploying daemon osd.10 on smithi047 2024-09-10T10:00:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:00:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:07 smithi162 ceph-mon[34963]: Deploying daemon osd.10 on smithi047 2024-09-10T10:00:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T10:00:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:07 smithi045 ceph-mon[31114]: Deploying daemon osd.10 on smithi047 2024-09-10T10:00:08.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:08 smithi162 ceph-mon[34963]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:08 smithi045 ceph-mon[31114]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:08.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:08 smithi047 ceph-mon[35244]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:10 smithi162 ceph-mon[34963]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:10.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:10 smithi045 ceph-mon[31114]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:10.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:10 smithi047 ceph-mon[35244]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:12.636 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:12 smithi047 ceph-mon[35244]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:12.636 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:12.636 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:12.636 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:12 smithi162 ceph-mon[34963]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:12.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:12 smithi045 ceph-mon[31114]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:12.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:12.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:12.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:14.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:14 smithi047 ceph-mon[35244]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:14.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:14 smithi162 ceph-mon[34963]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:14.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:14 smithi045 ceph-mon[31114]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:15.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.441+0000 7f4d177fe640 1 -- 172.21.15.47:0/2538893731 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4d2006a6f0 con 0x7f4d0c0a2bb0 2024-09-10T10:00:15.443 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 10 on host 'smithi047' 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d0c0a2bb0 msgr2=0x7f4d0c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d0c0a2bb0 0x7f4d0c0a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4d080314e0 tx=0x7f4d08000bc0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20100ad0 msgr2=0x7f4d20197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20100ad0 0x7f4d20197f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e9c0 tx=0x7f4d1000ee80 comp rx=0 tx=0).stop 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 shutdown_connections 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4d0c0a2bb0 0x7f4d0c0a5070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4d20102690 0x7f4d2019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d20101cd0 0x7f4d20198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 --2- 172.21.15.47:0/2538893731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d20100ad0 0x7f4d20197f40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.443+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 >> 172.21.15.47:0/2538893731 conn(0x7f4d200fc280 msgr2=0x7f4d20069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.444+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 shutdown_connections 2024-09-10T10:00:15.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-09-10T10:00:15.444+0000 7f4d24ddc640 1 -- 172.21.15.47:0/2538893731 wait complete. 2024-09-10T10:00:16.115 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:16.115 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.115 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.116 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:16.116 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:16.116 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.116 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:16.116 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.116 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.143 DEBUG:teuthology.orchestra.run.smithi047:osd.10> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T10:00:16.146 INFO:tasks.cephadm:Deploying osd.11 on smithi162 with /dev/vg_nvme/lv_1... 2024-09-10T10:00:16.146 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T10:00:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:16.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:16.405 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:00:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:17 smithi162 ceph-mon[34963]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:17 smithi045 ceph-mon[31114]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:18.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:17 smithi047 ceph-mon[35244]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:18.476 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:00:18.382+0000 7f9e31f00640 -1 osd.10 0 log_to_monitors true 2024-09-10T10:00:18.954 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:00:18.954 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:00:18.954 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:18.954 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:18.954 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:18.954 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:18.955 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:18.955 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:18.955 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:18.955 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:18.955 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:18 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:18.958 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T10:00:18.985 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_1 2024-09-10T10:00:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:00:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:00:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:19.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:19.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:18 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:19.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:19.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:18 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:19.236 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 -- 172.21.15.162:0/1834874402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 msgr2=0x7f3f5c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 --2- 172.21.15.162:0/1834874402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c100ed0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009900 tx=0x7f3f5802a410 comp rx=0 tx=0).stop 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 -- 172.21.15.162:0/1834874402 shutdown_connections 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 --2- 172.21.15.162:0/1834874402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c102690 0x7f3f5c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 --2- 172.21.15.162:0/1834874402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5c101cd0 0x7f3f5c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 --2- 172.21.15.162:0/1834874402 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c100ed0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 -- 172.21.15.162:0/1834874402 >> 172.21.15.162:0/1834874402 conn(0x7f3f5c0fc280 msgr2=0x7f3f5c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 -- 172.21.15.162:0/1834874402 shutdown_connections 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.655+0000 7f3f643bb640 1 -- 172.21.15.162:0/1834874402 wait complete. 2024-09-10T10:00:19.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 Processor -- start 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 -- start start 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5c101cd0 0x7f3f5c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c102690 0x7f3f5c19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10c760 con 0x7f3f5c101cd0 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10c5e0 con 0x7f3f5c102690 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f643bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c10c8e0 con 0x7f3f5c100ad0 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:60038/0 (socket says 172.21.15.162:60038) 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 -- 172.21.15.162:0/2773274173 learned_addr learned my addr 172.21.15.162:0/2773274173 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62958640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c102690 0x7f3f5c19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 -- 172.21.15.162:0/2773274173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c102690 msgr2=0x7f3f5c19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:19.684 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c102690 0x7f3f5c19f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 -- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5c101cd0 msgr2=0x7f3f5c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5c101cd0 0x7f3f5c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 -- 172.21.15.162:0/2773274173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c19fb10 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.656+0000 7f3f62157640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c197ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f58031610 tx=0x7f3f58004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.657+0000 7f3f537fe640 1 -- 172.21.15.162:0/2773274173 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5803f030 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.657+0000 7f3f537fe640 1 -- 172.21.15.162:0/2773274173 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f58003ad0 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.657+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c19fda0 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.657+0000 7f3f537fe640 1 -- 172.21.15.162:0/2773274173 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f58035070 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.657+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c1a0220 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.658+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c1a0450 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.659+0000 7f3f537fe640 1 -- 172.21.15.162:0/2773274173 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3f58042030 con 0x7f3f5c100ad0 2024-09-10T10:00:19.685 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.660+0000 7f3f537fe640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f3f480a2bb0 0x7f3f480a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.660+0000 7f3f537fe640 1 -- 172.21.15.162:0/2773274173 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6398+0+0 (secure 0 0 0) 0x7f3f580ef290 con 0x7f3f5c100ad0 2024-09-10T10:00:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.660+0000 7f3f61956640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f3f480a2bb0 0x7f3f480a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.660+0000 7f3f61956640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f3f480a2bb0 0x7f3f480a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c199400 tx=0x7f3f4c005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:19.686 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.663+0000 7f3f537fe640 1 -- 172.21.15.162:0/2773274173 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f580bc970 con 0x7f3f5c100ad0 2024-09-10T10:00:19.839 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:19.837+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3f5c1a06e0 con 0x7f3f480a2bb0 2024-09-10T10:00:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi047 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi047 to 4273M 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:00:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:19 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi047 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi047 to 4273M 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:00:20.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:00:20.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:19 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:20.160 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:00:19.871+0000 7f9e2e697640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T10:00:20.160 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:00:19.886+0000 7f9e29c9a640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:00:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi047 2024-09-10T10:00:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi047 to 4273M 2024-09-10T10:00:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T10:00:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T10:00:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: osdmap e66: 11 total, 10 up, 11 in 2024-09-10T10:00:20.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:20.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-09-10T10:00:20.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T10:00:20.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T10:00:20.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:19 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:21.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:20 smithi162 ceph-mon[34963]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:00:21.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:20 smithi162 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T10:00:21.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:20 smithi162 ceph-mon[34963]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T10:00:21.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:21.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:21.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:20 smithi045 ceph-mon[31114]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:00:21.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:20 smithi045 ceph-mon[31114]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T10:00:21.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:20 smithi045 ceph-mon[31114]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T10:00:21.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:21.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:21.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:20 smithi047 ceph-mon[35244]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:00:21.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:20 smithi047 ceph-mon[35244]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-09-10T10:00:21.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:20 smithi047 ceph-mon[35244]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T10:00:21.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:21.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:21 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:21 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:21 smithi162 ceph-mon[34963]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:21 smithi162 ceph-mon[34963]: osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] boot 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:21 smithi162 ceph-mon[34963]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:21 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:21 smithi162 ceph-mon[34963]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:21 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T10:00:22.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:21 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T10:00:22.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:21 smithi045 ceph-mon[31114]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:22.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:21 smithi045 ceph-mon[31114]: osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] boot 2024-09-10T10:00:22.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:21 smithi045 ceph-mon[31114]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T10:00:22.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:21 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:22.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:21 smithi045 ceph-mon[31114]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T10:00:22.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:21 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T10:00:22.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:21 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T10:00:22.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:21 smithi047 ceph-mon[35244]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-10T10:00:22.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:21 smithi047 ceph-mon[35244]: osd.10 [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] boot 2024-09-10T10:00:22.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:21 smithi047 ceph-mon[35244]: osdmap e68: 11 total, 11 up, 11 in 2024-09-10T10:00:22.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:21 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:00:22.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:21 smithi047 ceph-mon[35244]: osdmap e69: 11 total, 11 up, 11 in 2024-09-10T10:00:23.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:22 smithi162 ceph-mon[34963]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T10:00:23.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:22 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/1222420854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]: dispatch 2024-09-10T10:00:23.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:22 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]: dispatch 2024-09-10T10:00:23.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:22 smithi162 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]': finished 2024-09-10T10:00:23.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:22 smithi162 ceph-mon[34963]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T10:00:23.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:23.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:22 smithi162 ceph-mon[34963]: from='client.? 172.21.15.162:0/1775768202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:00:23.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:22 smithi045 ceph-mon[31114]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T10:00:23.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:22 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/1222420854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]: dispatch 2024-09-10T10:00:23.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:22 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]: dispatch 2024-09-10T10:00:23.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:22 smithi045 ceph-mon[31114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]': finished 2024-09-10T10:00:23.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:22 smithi045 ceph-mon[31114]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T10:00:23.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:23.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:22 smithi045 ceph-mon[31114]: from='client.? 172.21.15.162:0/1775768202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:00:23.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:22 smithi047 ceph-mon[35244]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T10:00:23.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:22 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/1222420854' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]: dispatch 2024-09-10T10:00:23.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:22 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]: dispatch 2024-09-10T10:00:23.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:22 smithi047 ceph-mon[35244]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "147281ba-9297-46a4-b3a1-c12eaa3456b3"}]': finished 2024-09-10T10:00:23.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:22 smithi047 ceph-mon[35244]: osdmap e71: 12 total, 11 up, 12 in 2024-09-10T10:00:23.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:23.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:22 smithi047 ceph-mon[35244]: from='client.? 172.21.15.162:0/1775768202' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T10:00:24.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:24 smithi162 ceph-mon[34963]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:24.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:24 smithi162 ceph-mon[34963]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:00:24.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:24 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:24.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:24 smithi045 ceph-mon[31114]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:24.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:24 smithi045 ceph-mon[31114]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:00:24.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:24 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:24.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:24 smithi047 ceph-mon[35244]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:24.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:24 smithi047 ceph-mon[35244]: osdmap e72: 12 total, 11 up, 12 in 2024-09-10T10:00:24.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:24 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:26 smithi162 ceph-mon[34963]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-09-10T10:00:26.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:26 smithi045 ceph-mon[31114]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-09-10T10:00:26.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:26 smithi047 ceph-mon[35244]: pgmap v230: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-09-10T10:00:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:28 smithi162 ceph-mon[34963]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-10T10:00:28.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:28 smithi045 ceph-mon[31114]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-10T10:00:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:28 smithi047 ceph-mon[35244]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-10T10:00:30.645 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:30 smithi162 ceph-mon[34963]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-10T10:00:30.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:30 smithi045 ceph-mon[31114]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-10T10:00:30.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:30 smithi047 ceph-mon[35244]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-10T10:00:31.588 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:00:31.588 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:00:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T10:00:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:32.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:32 smithi162 ceph-mon[34963]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T10:00:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:32 smithi162 ceph-mon[34963]: Deploying daemon osd.11 on smithi162 2024-09-10T10:00:32.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:32 smithi045 ceph-mon[31114]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T10:00:32.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:32 smithi045 ceph-mon[31114]: Deploying daemon osd.11 on smithi162 2024-09-10T10:00:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:32 smithi047 ceph-mon[35244]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-10T10:00:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:32 smithi047 ceph-mon[35244]: Deploying daemon osd.11 on smithi162 2024-09-10T10:00:34.576 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:34 smithi162 ceph-mon[34963]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:00:34.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:34 smithi045 ceph-mon[31114]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:00:34.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:34 smithi047 ceph-mon[35244]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T10:00:36.641 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:36 smithi162 ceph-mon[34963]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:36.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:36 smithi045 ceph-mon[31114]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:36 smithi047 ceph-mon[35244]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:37.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:37.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:37.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:37.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:38.622 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:38 smithi162 ceph-mon[34963]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:38 smithi045 ceph-mon[31114]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:38.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:38 smithi047 ceph-mon[35244]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:40.267 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 11 on host 'smithi162' 2024-09-10T10:00:40.268 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.265+0000 7f3f537fe640 1 -- 172.21.15.162:0/2773274173 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3f5c1a06e0 con 0x7f3f480a2bb0 2024-09-10T10:00:40.269 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f3f480a2bb0 msgr2=0x7f3f480a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f3f480a2bb0 0x7f3f480a5070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c199400 tx=0x7f3f4c005bf0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 msgr2=0x7f3f5c197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c197ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f58031610 tx=0x7f3f58004270 comp rx=0 tx=0).stop 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 shutdown_connections 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f3f480a2bb0 0x7f3f480a5070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f5c102690 0x7f3f5c19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f5c101cd0 0x7f3f5c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 --2- 172.21.15.162:0/2773274173 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3f5c100ad0 0x7f3f5c197ee0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 >> 172.21.15.162:0/2773274173 conn(0x7f3f5c0fc280 msgr2=0x7f3f5c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 shutdown_connections 2024-09-10T10:00:40.270 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:40.268+0000 7f3f643bb640 1 -- 172.21.15.162:0/2773274173 wait complete. 2024-09-10T10:00:40.573 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:40 smithi162 ceph-mon[34963]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:40.574 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:40 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:40.574 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:40 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:40.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:40 smithi045 ceph-mon[31114]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:40.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:40 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:40.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:40 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:40.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:40 smithi047 ceph-mon[35244]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:40.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:40 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:40.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:40 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:40.877 DEBUG:teuthology.orchestra.run.smithi162:osd.11> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T10:00:40.880 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T10:00:40.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:00:41.153 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:41.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.682+0000 7fef03030640 1 -- 172.21.15.45:0/914126250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc100ad0 msgr2=0x7feefc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:41.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 --2- 172.21.15.45:0/914126250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc100ad0 0x7feefc100ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7feef0009900 tx=0x7feef002a410 comp rx=0 tx=0).stop 2024-09-10T10:00:41.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 -- 172.21.15.45:0/914126250 shutdown_connections 2024-09-10T10:00:41.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 --2- 172.21.15.45:0/914126250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feefc102690 0x7feefc108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 --2- 172.21.15.45:0/914126250 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc101cd0 0x7feefc102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 --2- 172.21.15.45:0/914126250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc100ad0 0x7feefc100ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 -- 172.21.15.45:0/914126250 >> 172.21.15.45:0/914126250 conn(0x7feefc0fc280 msgr2=0x7feefc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 -- 172.21.15.45:0/914126250 shutdown_connections 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 -- 172.21.15.45:0/914126250 wait complete. 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 Processor -- start 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.683+0000 7fef03030640 1 -- start start 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef03030640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feefc100ad0 0x7feefc06aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef03030640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc101cd0 0x7feefc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef03030640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc102690 0x7feefc19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef03030640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feefc10c750 con 0x7feefc101cd0 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef03030640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feefc10c5d0 con 0x7feefc100ad0 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef03030640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7feefc10c8d0 con 0x7feefc102690 2024-09-10T10:00:41.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc101cd0 0x7feefc19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc101cd0 0x7feefc19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46264/0 (socket says 172.21.15.45:46264) 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 -- 172.21.15.45:0/2318665975 learned_addr learned my addr 172.21.15.45:0/2318665975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef015cd640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc102690 0x7feefc19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef00dcc640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feefc100ad0 0x7feefc06aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 -- 172.21.15.45:0/2318665975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc102690 msgr2=0x7feefc19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc102690 0x7feefc19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 -- 172.21.15.45:0/2318665975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feefc100ad0 msgr2=0x7feefc06aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feefc100ad0 0x7feefc06aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 -- 172.21.15.45:0/2318665975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc19fb60 con 0x7feefc101cd0 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feefbfff640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc101cd0 0x7feefc19cc30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feee800d770 tx=0x7feee800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feef9ffb640 1 -- 172.21.15.45:0/2318665975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8017070 con 0x7feefc101cd0 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7feef9ffb640 1 -- 172.21.15.45:0/2318665975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee8005500 con 0x7feefc101cd0 2024-09-10T10:00:41.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.684+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc19fdf0 con 0x7feefc101cd0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.685+0000 7feef9ffb640 1 -- 172.21.15.45:0/2318665975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8004e00 con 0x7feefc101cd0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.685+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc1a0250 con 0x7feefc101cd0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.686+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc107ed0 con 0x7feefc101cd0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.687+0000 7feef9ffb640 1 -- 172.21.15.45:0/2318665975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feee8004fa0 con 0x7feefc101cd0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.688+0000 7feef9ffb640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7feeec0a2b80 0x7feeec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.688+0000 7fef00dcc640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7feeec0a2b80 0x7feeec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.688+0000 7feef9ffb640 1 -- 172.21.15.45:0/2318665975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7feee80ce480 con 0x7feefc101cd0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.688+0000 7fef00dcc640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7feeec0a2b80 0x7feeec0a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feef0005ab0 tx=0x7feef0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:41.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.693+0000 7feef9ffb640 1 -- 172.21.15.45:0/2318665975 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feee809b2d0 con 0x7feefc101cd0 2024-09-10T10:00:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.872+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7feefc062e90 con 0x7feefc101cd0 2024-09-10T10:00:41.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.873+0000 7feef9ffb640 1 -- 172.21.15.45:0/2318665975 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7feee809f5d0 con 0x7feefc101cd0 2024-09-10T10:00:41.874 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7feeec0a2b80 msgr2=0x7feeec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7feeec0a2b80 0x7feeec0a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feef0005ab0 tx=0x7feef0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc101cd0 msgr2=0x7feefc19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc101cd0 0x7feefc19cc30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feee800d770 tx=0x7feee800dc30 comp rx=0 tx=0).stop 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 shutdown_connections 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7feeec0a2b80 0x7feeec0a5040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7feefc102690 0x7feefc19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feefc101cd0 0x7feefc19cc30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 --2- 172.21.15.45:0/2318665975 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feefc100ad0 0x7feefc06aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:41.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.875+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 >> 172.21.15.45:0/2318665975 conn(0x7feefc0fc280 msgr2=0x7feefc1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:41.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.876+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 shutdown_connections 2024-09-10T10:00:41.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:41.876+0000 7fef03030640 1 -- 172.21.15.45:0/2318665975 wait complete. 2024-09-10T10:00:42.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:42 smithi045 ceph-mon[31114]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:42.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:42.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:42 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:42.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:42 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2318665975' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:42.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:42 smithi162 ceph-mon[34963]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:42.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:42.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:42 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:42.389 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:42 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2318665975' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:42.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:42 smithi047 ceph-mon[35244]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:42.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:42.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:42 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:42.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:42 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2318665975' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:42.748 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1725962420,"num_in_osds":12,"osd_in_since":1725962422,"num_remapped_pgs":0} 2024-09-10T10:00:43.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:00:43 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:00:43.266+0000 7ff704aca640 -1 osd.11 0 log_to_monitors true 2024-09-10T10:00:43.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:00:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:44.140 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:44 smithi045 ceph-mon[31114]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:44.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:44 smithi045 ceph-mon[31114]: from='osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:00:44.141 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:44 smithi045 ceph-mon[31114]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:00:44.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:44 smithi162 ceph-mon[34963]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:44.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:44 smithi162 ceph-mon[34963]: from='osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:00:44.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:44 smithi162 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:00:44.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:44 smithi047 ceph-mon[35244]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:44.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:44 smithi047 ceph-mon[35244]: from='osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:00:44.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:44 smithi047 ceph-mon[35244]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T10:00:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 -- 172.21.15.45:0/3925662492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840101cd0 msgr2=0x7f3840102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 --2- 172.21.15.45:0/3925662492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840101cd0 0x7f3840102150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3834009900 tx=0x7f383402a410 comp rx=0 tx=0).stop 2024-09-10T10:00:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 -- 172.21.15.45:0/3925662492 shutdown_connections 2024-09-10T10:00:44.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 --2- 172.21.15.45:0/3925662492 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3840102690 0x7f3840108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 --2- 172.21.15.45:0/3925662492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840101cd0 0x7f3840102150 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 --2- 172.21.15.45:0/3925662492 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3840100ad0 0x7f3840100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 -- 172.21.15.45:0/3925662492 >> 172.21.15.45:0/3925662492 conn(0x7f38400fc280 msgr2=0x7f38400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 -- 172.21.15.45:0/3925662492 shutdown_connections 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.481+0000 7f3844863640 1 -- 172.21.15.45:0/3925662492 wait complete. 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 Processor -- start 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 -- start start 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840100ad0 0x7f3840104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3840101cd0 0x7f38401030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3840102690 0x7f38401035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f384010c730 con 0x7f3840100ad0 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f384010c5b0 con 0x7f3840101cd0 2024-09-10T10:00:44.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f3844863640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f384010c8b0 con 0x7f3840102690 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840100ad0 0x7f3840104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840100ad0 0x7f3840104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41574/0 (socket says 172.21.15.45:41574) 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 -- 172.21.15.45:0/1685685567 learned_addr learned my addr 172.21.15.45:0/1685685567 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383dd9b640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3840101cd0 0x7f38401030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383ed9d640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3840102690 0x7f38401035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 -- 172.21.15.45:0/1685685567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3840102690 msgr2=0x7f38401035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3840102690 0x7f38401035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 -- 172.21.15.45:0/1685685567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3840101cd0 msgr2=0x7f38401030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3840101cd0 0x7f38401030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 -- 172.21.15.45:0/1685685567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3840103eb0 con 0x7f3840100ad0 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383ed9d640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3840102690 0x7f38401035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:44.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.482+0000 7f383e59c640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840100ad0 0x7f3840104a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f383000c970 tx=0x7f383000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.483+0000 7f382f7fe640 1 -- 172.21.15.45:0/1685685567 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830007c10 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.483+0000 7f382f7fe640 1 -- 172.21.15.45:0/1685685567 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3830007db0 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.483+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38401a1f70 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.483+0000 7f382f7fe640 1 -- 172.21.15.45:0/1685685567 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383000f940 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.483+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401a2360 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.484+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38401a25f0 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.485+0000 7f382f7fe640 1 -- 172.21.15.45:0/1685685567 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f383000fae0 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.486+0000 7f382f7fe640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38280a2b80 0x7f38280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.486+0000 7f383dd9b640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38280a2b80 0x7f38280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.486+0000 7f382f7fe640 1 -- 172.21.15.45:0/1685685567 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6831+0+0 (secure 0 0 0) 0x7f38300cf0b0 con 0x7f3840100ad0 2024-09-10T10:00:44.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.486+0000 7f383dd9b640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38280a2b80 0x7f38280a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38340314e0 tx=0x7f3834000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:44.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.490+0000 7f382f7fe640 1 -- 172.21.15.45:0/1685685567 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f383009c700 con 0x7f3840100ad0 2024-09-10T10:00:44.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.667+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f38401a27c0 con 0x7f3840100ad0 2024-09-10T10:00:44.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.667+0000 7f382f7fe640 1 -- 172.21.15.45:0/1685685567 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f383009c000 con 0x7f3840100ad0 2024-09-10T10:00:44.669 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:00:44.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38280a2b80 msgr2=0x7f38280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:44.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38280a2b80 0x7f38280a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f38340314e0 tx=0x7f3834000bc0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840100ad0 msgr2=0x7f3840104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840100ad0 0x7f3840104a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f383000c970 tx=0x7f383000ce30 comp rx=0 tx=0).stop 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 shutdown_connections 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38280a2b80 0x7f38280a5040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3840102690 0x7f38401035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3840101cd0 0x7f38401030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 --2- 172.21.15.45:0/1685685567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3840100ad0 0x7f3840104a60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 >> 172.21.15.45:0/1685685567 conn(0x7f38400fc280 msgr2=0x7f38400fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 shutdown_connections 2024-09-10T10:00:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:44.670+0000 7f3844863640 1 -- 172.21.15.45:0/1685685567 wait complete. 2024-09-10T10:00:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:00:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T10:00:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T10:00:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T10:00:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: Detected new or changed devices on smithi162 2024-09-10T10:00:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi162 to 4272M 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:45.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1685685567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: Detected new or changed devices on smithi162 2024-09-10T10:00:45.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi162 to 4272M 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:45.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:45.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:45 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1685685567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: osdmap e73: 12 total, 11 up, 12 in 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: Detected new or changed devices on smithi162 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi162 to 4272M 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:45.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:45 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1685685567' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:45.563 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1725962420,"num_in_osds":12,"osd_in_since":1725962422,"num_remapped_pgs":0} 2024-09-10T10:00:46.149 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:00:45.787+0000 7ff70022a640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T10:00:46.150 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:00:45 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:00:45.801+0000 7ff6f885c640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:00:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:46 smithi162 ceph-mon[34963]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:46 smithi162 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T10:00:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:46 smithi162 ceph-mon[34963]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T10:00:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:46 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:46 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:46 smithi162 ceph-mon[34963]: from='osd.11 ' entity='osd.11' 2024-09-10T10:00:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:46 smithi045 ceph-mon[31114]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:46 smithi045 ceph-mon[31114]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T10:00:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:46 smithi045 ceph-mon[31114]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T10:00:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:46 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:46.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:46 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:46.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:46 smithi045 ceph-mon[31114]: from='osd.11 ' entity='osd.11' 2024-09-10T10:00:46.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:46 smithi047 ceph-mon[35244]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:46.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:46 smithi047 ceph-mon[35244]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T10:00:46.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:46 smithi047 ceph-mon[35244]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T10:00:46.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:46 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:46.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:46 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:46.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:46 smithi047 ceph-mon[35244]: from='osd.11 ' entity='osd.11' 2024-09-10T10:00:46.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T10:00:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:47.298 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:47 smithi045 ceph-mon[31114]: purged_snaps scrub starts 2024-09-10T10:00:47.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:47 smithi045 ceph-mon[31114]: purged_snaps scrub ok 2024-09-10T10:00:47.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:47.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:47 smithi045 ceph-mon[31114]: osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] boot 2024-09-10T10:00:47.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:47 smithi045 ceph-mon[31114]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:00:47.299 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:47 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.380+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/301806837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c100de0 msgr2=0x7fb89c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.380+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/301806837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c100de0 0x7fb89c101260 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb888009900 tx=0x7fb88802a410 comp rx=0 tx=0).stop 2024-09-10T10:00:47.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.380+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/301806837 shutdown_connections 2024-09-10T10:00:47.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.380+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/301806837 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c1017a0 0x7fb89c109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.381+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/301806837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c100de0 0x7fb89c101260 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.381+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/301806837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb89c1020a0 0x7fb89c1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.381+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/301806837 >> 172.21.15.45:0/301806837 conn(0x7fb89c0fba40 msgr2=0x7fb89c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:47.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.381+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/301806837 shutdown_connections 2024-09-10T10:00:47.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.381+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/301806837 wait complete. 2024-09-10T10:00:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 Processor -- start 2024-09-10T10:00:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 -- start start 2024-09-10T10:00:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb89c100de0 0x7fb89c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c1017a0 0x7fb89c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c1020a0 0x7fb89c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10c7a0 con 0x7fb89c1020a0 2024-09-10T10:00:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10c620 con 0x7fb89c100de0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a2c6e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10c920 con 0x7fb89c1017a0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c1020a0 0x7fb89c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c1020a0 0x7fb89c1a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41580/0 (socket says 172.21.15.45:41580) 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 -- 172.21.15.45:0/1146804094 learned_addr learned my addr 172.21.15.45:0/1146804094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a0a0a640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb89c100de0 0x7fb89c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb89bfff640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c1017a0 0x7fb89c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 -- 172.21.15.45:0/1146804094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c1017a0 msgr2=0x7fb89c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c1017a0 0x7fb89c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 -- 172.21.15.45:0/1146804094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb89c100de0 msgr2=0x7fb89c19a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb89c100de0 0x7fb89c19a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.382+0000 7fb8a120b640 1 -- 172.21.15.45:0/1146804094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c1a1da0 con 0x7fb89c1020a0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.383+0000 7fb8a120b640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c1020a0 0x7fb89c1a1730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb89400cad0 tx=0x7fb89400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.383+0000 7fb899ffb640 1 -- 172.21.15.45:0/1146804094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89400ae20 con 0x7fb89c1020a0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.383+0000 7fb899ffb640 1 -- 172.21.15.45:0/1146804094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb894005020 con 0x7fb89c1020a0 2024-09-10T10:00:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.383+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c1a2090 con 0x7fb89c1020a0 2024-09-10T10:00:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.383+0000 7fb899ffb640 1 -- 172.21.15.45:0/1146804094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89400f6f0 con 0x7fb89c1020a0 2024-09-10T10:00:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.383+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c1a25a0 con 0x7fb89c1020a0 2024-09-10T10:00:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.384+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c1a2860 con 0x7fb89c1020a0 2024-09-10T10:00:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.385+0000 7fb899ffb640 1 -- 172.21.15.45:0/1146804094 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb8940051d0 con 0x7fb89c1020a0 2024-09-10T10:00:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.386+0000 7fb899ffb640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb88c0a2bb0 0x7fb88c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.386+0000 7fb8a0a0a640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb88c0a2bb0 0x7fb88c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:47.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.386+0000 7fb899ffb640 1 -- 172.21.15.45:0/1146804094 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb8940ceeb0 con 0x7fb89c1020a0 2024-09-10T10:00:47.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.386+0000 7fb8a0a0a640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb88c0a2bb0 0x7fb88c0a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb89c06a510 tx=0x7fb89000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:47.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.390+0000 7fb899ffb640 1 -- 172.21.15.45:0/1146804094 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb894014030 con 0x7fb89c1020a0 2024-09-10T10:00:47.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:47 smithi162 ceph-mon[34963]: purged_snaps scrub starts 2024-09-10T10:00:47.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:47 smithi162 ceph-mon[34963]: purged_snaps scrub ok 2024-09-10T10:00:47.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:47.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:47 smithi162 ceph-mon[34963]: osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] boot 2024-09-10T10:00:47.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:47 smithi162 ceph-mon[34963]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:00:47.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:47 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:47.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:47 smithi047 ceph-mon[35244]: purged_snaps scrub starts 2024-09-10T10:00:47.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:47 smithi047 ceph-mon[35244]: purged_snaps scrub ok 2024-09-10T10:00:47.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:47.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:47 smithi047 ceph-mon[35244]: osd.11 [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] boot 2024-09-10T10:00:47.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:47 smithi047 ceph-mon[35244]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T10:00:47.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:47 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:00:47.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.563+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb89c1a2a30 con 0x7fb89c1020a0 2024-09-10T10:00:47.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.564+0000 7fb899ffb640 1 -- 172.21.15.45:0/1146804094 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fb89409bbc0 con 0x7fb89c1020a0 2024-09-10T10:00:47.565 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:00:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb88c0a2bb0 msgr2=0x7fb88c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb88c0a2bb0 0x7fb88c0a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb89c06a510 tx=0x7fb89000b390 comp rx=0 tx=0).stop 2024-09-10T10:00:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c1020a0 msgr2=0x7fb89c1a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:47.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c1020a0 0x7fb89c1a1730 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb89400cad0 tx=0x7fb89400cf90 comp rx=0 tx=0).stop 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 shutdown_connections 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb88c0a2bb0 0x7fb88c0a5070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89c1020a0 0x7fb89c1a1730 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb89c1017a0 0x7fb89c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 --2- 172.21.15.45:0/1146804094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb89c100de0 0x7fb89c19a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 >> 172.21.15.45:0/1146804094 conn(0x7fb89c0fba40 msgr2=0x7fb89c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 shutdown_connections 2024-09-10T10:00:47.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:47.566+0000 7fb8a2c6e640 1 -- 172.21.15.45:0/1146804094 wait complete. 2024-09-10T10:00:48.258 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:48 smithi045 ceph-mon[31114]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:48.258 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:48 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1146804094' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:48.259 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1725962446,"num_in_osds":12,"osd_in_since":1725962422,"num_remapped_pgs":0} 2024-09-10T10:00:48.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:00:48.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:48 smithi162 ceph-mon[34963]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:48.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:48 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1146804094' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:48.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:48 smithi047 ceph-mon[35244]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-10T10:00:48.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:48 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1146804094' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T10:00:48.514 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 -- 172.21.15.45:0/1491754119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20106a80 msgr2=0x7fda20106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 --2- 172.21.15.45:0/1491754119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20106a80 0x7fda20106e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fda10009930 tx=0x7fda1002a420 comp rx=0 tx=0).stop 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 -- 172.21.15.45:0/1491754119 shutdown_connections 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 --2- 172.21.15.45:0/1491754119 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda20101420 0x7fda2010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 --2- 172.21.15.45:0/1491754119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda20100a80 0x7fda20100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 --2- 172.21.15.45:0/1491754119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20106a80 0x7fda20106e60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 -- 172.21.15.45:0/1491754119 >> 172.21.15.45:0/1491754119 conn(0x7fda200fc7a0 msgr2=0x7fda200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 -- 172.21.15.45:0/1491754119 shutdown_connections 2024-09-10T10:00:49.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.070+0000 7fda250f4640 1 -- 172.21.15.45:0/1491754119 wait complete. 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 Processor -- start 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 -- start start 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda20100a80 0x7fda201983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20101420 0x7fda20198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda20106a80 0x7fda2019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fda20110e40 con 0x7fda20101420 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fda20110cc0 con 0x7fda20100a80 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.071+0000 7fda250f4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fda20110fc0 con 0x7fda20106a80 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20101420 0x7fda20198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20101420 0x7fda20198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41602/0 (socket says 172.21.15.45:41602) 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 -- 172.21.15.45:0/2505490043 learned_addr learned my addr 172.21.15.45:0/2505490043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:00:49.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1ed9d640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda20100a80 0x7fda201983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1f59e640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda20106a80 0x7fda2019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 -- 172.21.15.45:0/2505490043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda20106a80 msgr2=0x7fda2019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda20106a80 0x7fda2019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 -- 172.21.15.45:0/2505490043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda20100a80 msgr2=0x7fda201983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda20100a80 0x7fda201983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 -- 172.21.15.45:0/2505490043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2019d370 con 0x7fda20101420 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda1e59c640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20101420 0x7fda20198900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c920 tx=0x7fda0800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda17fff640 1 -- 172.21.15.45:0/2505490043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08007c70 con 0x7fda20101420 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda17fff640 1 -- 172.21.15.45:0/2505490043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda08007e10 con 0x7fda20101420 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2019d660 con 0x7fda20101420 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda17fff640 1 -- 172.21.15.45:0/2505490043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda08018890 con 0x7fda20101420 2024-09-10T10:00:49.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.072+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda201a4f40 con 0x7fda20101420 2024-09-10T10:00:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.073+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda20100ee0 con 0x7fda20101420 2024-09-10T10:00:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.075+0000 7fda17fff640 1 -- 172.21.15.45:0/2505490043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fda08018a70 con 0x7fda20101420 2024-09-10T10:00:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.075+0000 7fda17fff640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fda0c0a2b80 0x7fda0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.076+0000 7fda1ed9d640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fda0c0a2b80 0x7fda0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.076+0000 7fda17fff640 1 -- 172.21.15.45:0/2505490043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fda080cecf0 con 0x7fda20101420 2024-09-10T10:00:49.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.076+0000 7fda1ed9d640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fda0c0a2b80 0x7fda0c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda10009900 tx=0x7fda100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:49.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.079+0000 7fda17fff640 1 -- 172.21.15.45:0/2505490043 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda080d3070 con 0x7fda20101420 2024-09-10T10:00:49.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.257+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda2002d7c0 con 0x7fda20101420 2024-09-10T10:00:49.259 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:49 smithi045 ceph-mon[31114]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:00:49.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.258+0000 7fda17fff640 1 -- 172.21.15.45:0/2505490043 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18431 (secure 0 0 0) 0x7fda08014030 con 0x7fda20101420 2024-09-10T10:00:49.260 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:00:49.260 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":76,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:00:48.030150+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:00:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.260+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fda0c0a2b80 msgr2=0x7fda0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.260+0000 7fda250f4640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fda0c0a2b80 0x7fda0c0a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fda10009900 tx=0x7fda100058d0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.260+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20101420 msgr2=0x7fda20198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.260+0000 7fda250f4640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20101420 0x7fda20198900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fda0800c920 tx=0x7fda0800cde0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 shutdown_connections 2024-09-10T10:00:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fda0c0a2b80 0x7fda0c0a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda20106a80 0x7fda2019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda20101420 0x7fda20198900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 --2- 172.21.15.45:0/2505490043 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda20100a80 0x7fda201983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 >> 172.21.15.45:0/2505490043 conn(0x7fda200fc7a0 msgr2=0x7fda2006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 shutdown_connections 2024-09-10T10:00:49.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:49.261+0000 7fda250f4640 1 -- 172.21.15.45:0/2505490043 wait complete. 2024-09-10T10:00:49.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:49 smithi162 ceph-mon[34963]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:00:49.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:49 smithi047 ceph-mon[35244]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T10:00:49.858 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:57:03.045230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:00:49.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T10:00:50.114 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:50 smithi045 ceph-mon[31114]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:50 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2505490043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:00:50.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:50 smithi162 ceph-mon[34963]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:50.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:50 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2505490043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:00:50.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:50 smithi047 ceph-mon[35244]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:50.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:50 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2505490043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:00:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 -- 172.21.15.45:0/328031511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48102d30 msgr2=0x7faa48103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 --2- 172.21.15.45:0/328031511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48102d30 0x7faa48103110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faa38009930 tx=0x7faa3802a420 comp rx=0 tx=0).stop 2024-09-10T10:00:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 -- 172.21.15.45:0/328031511 shutdown_connections 2024-09-10T10:00:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 --2- 172.21.15.45:0/328031511 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa481082a0 0x7faa4810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 --2- 172.21.15.45:0/328031511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa481036e0 0x7faa48107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 --2- 172.21.15.45:0/328031511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa48102d30 0x7faa48103110 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 -- 172.21.15.45:0/328031511 >> 172.21.15.45:0/328031511 conn(0x7faa480fc7d0 msgr2=0x7faa480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 -- 172.21.15.45:0/328031511 shutdown_connections 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.608+0000 7faa4d88b640 1 -- 172.21.15.45:0/328031511 wait complete. 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 Processor -- start 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 -- start start 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa48102d30 0x7faa48100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa481036e0 0x7faa48100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481082a0 0x7faa48101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa48112100 con 0x7faa481082a0 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa48111f80 con 0x7faa48102d30 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa4d88b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7faa48112280 con 0x7faa481036e0 2024-09-10T10:00:50.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481082a0 0x7faa48101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa467fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa481036e0 0x7faa48100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa477fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481082a0 0x7faa48101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41628/0 (socket says 172.21.15.45:41628) 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa477fe640 1 -- 172.21.15.45:0/6926431 learned_addr learned my addr 172.21.15.45:0/6926431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.609+0000 7faa46ffd640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa48102d30 0x7faa48100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa477fe640 1 -- 172.21.15.45:0/6926431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa481036e0 msgr2=0x7faa48100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa477fe640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa481036e0 0x7faa48100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa477fe640 1 -- 172.21.15.45:0/6926431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa48102d30 msgr2=0x7faa48100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa477fe640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa48102d30 0x7faa48100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa477fe640 1 -- 172.21.15.45:0/6926431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa481a6a30 con 0x7faa481082a0 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa477fe640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481082a0 0x7faa48101410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b970 tx=0x7faa4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa4c889640 1 -- 172.21.15.45:0/6926431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40019070 con 0x7faa481082a0 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa4c889640 1 -- 172.21.15.45:0/6926431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa40014440 con 0x7faa481082a0 2024-09-10T10:00:50.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa4c889640 1 -- 172.21.15.45:0/6926431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa400075d0 con 0x7faa481082a0 2024-09-10T10:00:50.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa481a6d20 con 0x7faa481082a0 2024-09-10T10:00:50.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.610+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa481a7210 con 0x7faa481082a0 2024-09-10T10:00:50.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.611+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4806a850 con 0x7faa481082a0 2024-09-10T10:00:50.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.612+0000 7faa4c889640 1 -- 172.21.15.45:0/6926431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7faa4001c070 con 0x7faa481082a0 2024-09-10T10:00:50.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.613+0000 7faa4c889640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7faa340a2b80 0x7faa340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:50.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.613+0000 7faa4c889640 1 -- 172.21.15.45:0/6926431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7faa400ce660 con 0x7faa481082a0 2024-09-10T10:00:50.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.613+0000 7faa46ffd640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7faa340a2b80 0x7faa340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:50.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.614+0000 7faa46ffd640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7faa340a2b80 0x7faa340a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa38009900 tx=0x7faa380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:50.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.616+0000 7faa4c889640 1 -- 172.21.15.45:0/6926431 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa4009b370 con 0x7faa481082a0 2024-09-10T10:00:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.793+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa4810a2b0 con 0x7faa481082a0 2024-09-10T10:00:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.793+0000 7faa4c889640 1 -- 172.21.15.45:0/6926431 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7faa4009f670 con 0x7faa481082a0 2024-09-10T10:00:50.796 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7faa340a2b80 msgr2=0x7faa340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7faa340a2b80 0x7faa340a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa38009900 tx=0x7faa380058d0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481082a0 msgr2=0x7faa48101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481082a0 0x7faa48101410 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b970 tx=0x7faa4000bd70 comp rx=0 tx=0).stop 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 shutdown_connections 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7faa340a2b80 0x7faa340a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa481082a0 0x7faa48101410 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faa481036e0 0x7faa48100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 --2- 172.21.15.45:0/6926431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa48102d30 0x7faa48100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.795+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 >> 172.21.15.45:0/6926431 conn(0x7faa480fc7d0 msgr2=0x7faa48105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:50.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.796+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 shutdown_connections 2024-09-10T10:00:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:50.796+0000 7faa4d88b640 1 -- 172.21.15.45:0/6926431 wait complete. 2024-09-10T10:00:51.385 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:51 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/6926431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:00:51.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:51 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/6926431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:00:51.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:51 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/6926431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.a on smithi045 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.d on smithi045 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.g on smithi045 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.j on smithi045 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.b on smithi047 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.e on smithi047 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.h on smithi047 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.k on smithi047 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.c on smithi162 2024-09-10T10:00:51.427 INFO:tasks.cephadm:Adding mds.f on smithi162 2024-09-10T10:00:51.428 INFO:tasks.cephadm:Adding mds.i on smithi162 2024-09-10T10:00:51.428 INFO:tasks.cephadm:Adding mds.l on smithi162 2024-09-10T10:00:51.428 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l' 2024-09-10T10:00:51.687 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:00:52.118 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:52 smithi162 ceph-mon[34963]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:52.123 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.116+0000 7f24c8cb5640 1 -- 172.21.15.162:0/1948729737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4105030 msgr2=0x7f24c4107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.116+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/1948729737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4105030 0x7f24c4107420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009900 tx=0x7f24b402a410 comp rx=0 tx=0).stop 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.116+0000 7f24c8cb5640 1 -- 172.21.15.162:0/1948729737 shutdown_connections 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.116+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/1948729737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c4107960 0x7f24c4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.116+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/1948729737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4105030 0x7f24c4107420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.116+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/1948729737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c4102700 0x7f24c4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.116+0000 7f24c8cb5640 1 -- 172.21.15.162:0/1948729737 >> 172.21.15.162:0/1948729737 conn(0x7f24c40fc280 msgr2=0x7f24c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.117+0000 7f24c8cb5640 1 -- 172.21.15.162:0/1948729737 shutdown_connections 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.117+0000 7f24c8cb5640 1 -- 172.21.15.162:0/1948729737 wait complete. 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 Processor -- start 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 -- start start 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4102700 0x7f24c4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c4105030 0x7f24c4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.124 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c4107960 0x7f24c419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24c410d850 con 0x7f24c4105030 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f24c410d6d0 con 0x7f24c4107960 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c8cb5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f24c410d9d0 con 0x7f24c4102700 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c259c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4102700 0x7f24c4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c259c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4102700 0x7f24c4197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:32944/0 (socket says 172.21.15.162:32944) 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c259c640 1 -- 172.21.15.162:0/2712906284 learned_addr learned my addr 172.21.15.162:0/2712906284 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c2d9d640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c4107960 0x7f24c419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.118+0000 7f24c1d9b640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c4105030 0x7f24c4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c259c640 1 -- 172.21.15.162:0/2712906284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c4107960 msgr2=0x7f24c419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c259c640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c4107960 0x7f24c419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c259c640 1 -- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c4105030 msgr2=0x7f24c4198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.125 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c259c640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c4105030 0x7f24c4198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c259c640 1 -- 172.21.15.162:0/2712906284 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c419fb60 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c2d9d640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c4107960 0x7f24c419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c259c640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4102700 0x7f24c4197f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c890 tx=0x7f24ac00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24bb7fe640 1 -- 172.21.15.162:0/2712906284 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac004220 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24bb7fe640 1 -- 172.21.15.162:0/2712906284 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ac0043c0 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c419fe50 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24bb7fe640 1 -- 172.21.15.162:0/2712906284 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac0049c0 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.119+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41a02a0 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.120+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c40fdd70 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.121+0000 7f24bb7fe640 1 -- 172.21.15.162:0/2712906284 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f24ac004b60 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.122+0000 7f24bb7fe640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f24b00a2bb0 0x7f24b00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.123+0000 7f24bb7fe640 1 -- 172.21.15.162:0/2712906284 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f24ac0ce790 con 0x7f24c4102700 2024-09-10T10:00:52.126 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.123+0000 7f24c1d9b640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f24b00a2bb0 0x7f24b00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:52.127 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.123+0000 7f24c1d9b640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f24b00a2bb0 0x7f24b00a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24b4005ab0 tx=0x7f24b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:52.128 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.127+0000 7f24bb7fe640 1 -- 172.21.15.162:0/2712906284 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24ac014030 con 0x7f24c4102700 2024-09-10T10:00:52.312 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.310+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}) -- 0x7f24c4062730 con 0x7f24b00a2bb0 2024-09-10T10:00:52.320 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.318+0000 7f24bb7fe640 1 -- 172.21.15.162:0/2712906284 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f24c4062730 con 0x7f24b00a2bb0 2024-09-10T10:00:52.321 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mds.all update... 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f24b00a2bb0 msgr2=0x7f24b00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f24b00a2bb0 0x7f24b00a5070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f24b4005ab0 tx=0x7f24b4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4102700 msgr2=0x7f24c4197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4102700 0x7f24c4197f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c890 tx=0x7f24ac00cd50 comp rx=0 tx=0).stop 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 shutdown_connections 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f24b00a2bb0 0x7f24b00a5070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f24c4107960 0x7f24c419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24c4105030 0x7f24c4198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.322 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 --2- 172.21.15.162:0/2712906284 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24c4102700 0x7f24c4197f30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:52.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 >> 172.21.15.162:0/2712906284 conn(0x7f24c40fc280 msgr2=0x7f24c4102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:52.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 shutdown_connections 2024-09-10T10:00:52.323 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T10:00:52.320+0000 7f24c8cb5640 1 -- 172.21.15.162:0/2712906284 wait complete. 2024-09-10T10:00:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:52 smithi045 ceph-mon[31114]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:52.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:52 smithi047 ceph-mon[35244]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:53.082 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T10:00:53.085 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T10:00:53.128 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T10:00:53.171 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T10:00:53.215 DEBUG:teuthology.orchestra.run.smithi047:mds.b> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T10:00:53.217 DEBUG:teuthology.orchestra.run.smithi047:mds.e> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T10:00:53.260 DEBUG:teuthology.orchestra.run.smithi047:mds.h> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T10:00:53.304 DEBUG:teuthology.orchestra.run.smithi047:mds.k> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T10:00:53.347 DEBUG:teuthology.orchestra.run.smithi162:mds.c> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T10:00:53.350 DEBUG:teuthology.orchestra.run.smithi162:mds.f> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T10:00:53.394 DEBUG:teuthology.orchestra.run.smithi162:mds.i> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T10:00:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:00:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-09-10T10:00:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:53.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:53.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:53.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:53 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:53.437 DEBUG:teuthology.orchestra.run.smithi162:mds.l> sudo journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T10:00:53.480 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T10:00:53.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:00:53.606 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:53.607 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:53 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:53.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:53.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:53 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:53.731 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:54.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 -- 172.21.15.45:0/4135103045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 msgr2=0x7fab3810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 --2- 172.21.15.45:0/4135103045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3810eaa0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab30009ca0 tx=0x7fab3002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 -- 172.21.15.45:0/4135103045 shutdown_connections 2024-09-10T10:00:54.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 --2- 172.21.15.45:0/4135103045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3810eaa0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 --2- 172.21.15.45:0/4135103045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab381027a0 0x7fab381067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 --2- 172.21.15.45:0/4135103045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab38101df0 0x7fab381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 -- 172.21.15.45:0/4135103045 >> 172.21.15.45:0/4135103045 conn(0x7fab380fd8b0 msgr2=0x7fab380ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 -- 172.21.15.45:0/4135103045 shutdown_connections 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.185+0000 7fab3f05c640 1 -- 172.21.15.45:0/4135103045 wait complete. 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 Processor -- start 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 -- start start 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab38101df0 0x7fab381983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab381027a0 0x7fab38198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab38111fd0 con 0x7fab38106f20 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab38111e50 con 0x7fab38101df0 2024-09-10T10:00:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3f05c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fab38112150 con 0x7fab381027a0 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3cdf8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab38101df0 0x7fab381983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44114/0 (socket says 172.21.15.45:44114) 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 -- 172.21.15.45:0/152116501 learned_addr learned my addr 172.21.15.45:0/152116501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab37fff640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab381027a0 0x7fab38198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 -- 172.21.15.45:0/152116501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab381027a0 msgr2=0x7fab38198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab381027a0 0x7fab38198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 -- 172.21.15.45:0/152116501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab38101df0 msgr2=0x7fab381983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab38101df0 0x7fab381983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.186+0000 7fab3d5f9640 1 -- 172.21.15.45:0/152116501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3819d3a0 con 0x7fab38106f20 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.187+0000 7fab3d5f9640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3819ccc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab30003f10 tx=0x7fab30003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.187+0000 7fab35ffb640 1 -- 172.21.15.45:0/152116501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30031710 con 0x7fab38106f20 2024-09-10T10:00:54.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.187+0000 7fab35ffb640 1 -- 172.21.15.45:0/152116501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab300318b0 con 0x7fab38106f20 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.187+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3819d630 con 0x7fab38106f20 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.187+0000 7fab35ffb640 1 -- 172.21.15.45:0/152116501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3002fb50 con 0x7fab38106f20 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.187+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab381a4f70 con 0x7fab38106f20 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.188+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab381021d0 con 0x7fab38106f20 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.189+0000 7fab35ffb640 1 -- 172.21.15.45:0/152116501 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab3002fcf0 con 0x7fab38106f20 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.190+0000 7fab35ffb640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fab200a2b80 0x7fab200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.190+0000 7fab3cdf8640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fab200a2b80 0x7fab200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.190+0000 7fab35ffb640 1 -- 172.21.15.45:0/152116501 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fab300f0380 con 0x7fab38106f20 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.190+0000 7fab3cdf8640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fab200a2b80 0x7fab200a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab28005fd0 tx=0x7fab2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:54.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.193+0000 7fab35ffb640 1 -- 172.21.15.45:0/152116501 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab300bd6e0 con 0x7fab38106f20 2024-09-10T10:00:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:54 smithi045 ceph-mon[31114]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:54.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:54 smithi045 ceph-mon[31114]: Deploying daemon mds.l on smithi162 2024-09-10T10:00:54.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.407+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab38199750 con 0x7fab38106f20 2024-09-10T10:00:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.407+0000 7fab35ffb640 1 -- 172.21.15.45:0/152116501 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fab38199750 con 0x7fab38106f20 2024-09-10T10:00:54.410 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:00:54.410 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"btime":"2024-09-10T09:53:53:416702+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T10:00:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2024-09-10T10:00:54.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fab200a2b80 msgr2=0x7fab200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fab200a2b80 0x7fab200a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab28005fd0 tx=0x7fab2800b390 comp rx=0 tx=0).stop 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 msgr2=0x7fab3819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3819ccc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fab30003f10 tx=0x7fab30003f40 comp rx=0 tx=0).stop 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 shutdown_connections 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fab200a2b80 0x7fab200a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab38106f20 0x7fab3819ccc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fab381027a0 0x7fab38198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 --2- 172.21.15.45:0/152116501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab38101df0 0x7fab381983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 >> 172.21.15.45:0/152116501 conn(0x7fab380fd8b0 msgr2=0x7fab380feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 shutdown_connections 2024-09-10T10:00:54.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:54.410+0000 7fab3f05c640 1 -- 172.21.15.45:0/152116501 wait complete. 2024-09-10T10:00:54.506 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:54 smithi162 ceph-mon[34963]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:54.507 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:54 smithi162 ceph-mon[34963]: Deploying daemon mds.l on smithi162 2024-09-10T10:00:54.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:54 smithi047 ceph-mon[35244]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:54.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:54 smithi047 ceph-mon[35244]: Deploying daemon mds.l on smithi162 2024-09-10T10:00:54.900 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:54 smithi162 systemd[1]: Starting Ceph mds.l for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:00:55.223 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:54 smithi162 podman[63812]: 2024-09-10 10:00:54.927913418 +0000 UTC m=+0.096175184 container create 5564991b156ee7bcdc93fb920dd283bf72f66ed11bc1d5e0812608c9c4aea0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:00:55.223 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:54 smithi162 podman[63812]: 2024-09-10 10:00:54.847631494 +0000 UTC m=+0.015893259 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:00:55.223 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:55 smithi162 podman[63812]: 2024-09-10 10:00:55.13033483 +0000 UTC m=+0.298596595 container init 5564991b156ee7bcdc93fb920dd283bf72f66ed11bc1d5e0812608c9c4aea0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:00:55.223 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:55 smithi162 podman[63812]: 2024-09-10 10:00:55.133161926 +0000 UTC m=+0.301423699 container start 5564991b156ee7bcdc93fb920dd283bf72f66ed11bc1d5e0812608c9c4aea0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T10:00:55.223 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mds[63831]: -- 172.21.15.162:0/2592279673 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ddf0ef860 con 0x563ddf0cd400 2024-09-10T10:00:55.223 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l[63827]: starting mds.l at 2024-09-10T10:00:55.223 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:55 smithi162 bash[63812]: 5564991b156ee7bcdc93fb920dd283bf72f66ed11bc1d5e0812608c9c4aea0f3 2024-09-10T10:00:55.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/152116501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.162:6832/531637764,v1:172.21.15.162:6833/531637764] up:boot 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: fsmap 1 up:standby 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:55 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:55.649 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:00:55 smithi162 systemd[1]: Started Ceph mds.l for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/152116501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.162:6832/531637764,v1:172.21.15.162:6833/531637764] up:boot 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: fsmap 1 up:standby 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:55.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:55 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:55.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/152116501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:55.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.162:6832/531637764,v1:172.21.15.162:6833/531637764] up:boot 2024-09-10T10:00:55.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: fsmap 1 up:standby 2024-09-10T10:00:55.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:00:55.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:55.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:55.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:55.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:55 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:56.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:56 smithi162 ceph-mon[34963]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:56.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:56 smithi162 ceph-mon[34963]: Deploying daemon mds.k on smithi047 2024-09-10T10:00:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:56 smithi045 ceph-mon[31114]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:56.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:56 smithi045 ceph-mon[31114]: Deploying daemon mds.k on smithi047 2024-09-10T10:00:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:56 smithi047 ceph-mon[35244]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:56 smithi047 ceph-mon[35244]: Deploying daemon mds.k on smithi047 2024-09-10T10:00:57.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:00:57.343 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 systemd[1]: Starting Ceph mds.k for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:00:57.399 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:57.660 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 podman[65188]: 2024-09-10 10:00:57.244651502 +0000 UTC m=+0.015774336 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:00:57.660 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 podman[65188]: 2024-09-10 10:00:57.352400737 +0000 UTC m=+0.123523612 container create 1b558e67b92b6e7a5aa9e3da24299432c7c88ff4af97440e99cf665fbacfa305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T10:00:57.660 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 podman[65188]: 2024-09-10 10:00:57.554579765 +0000 UTC m=+0.325702599 container init 1b558e67b92b6e7a5aa9e3da24299432c7c88ff4af97440e99cf665fbacfa305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T10:00:57.660 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 podman[65188]: 2024-09-10 10:00:57.557599848 +0000 UTC m=+0.328722681 container start 1b558e67b92b6e7a5aa9e3da24299432c7c88ff4af97440e99cf665fbacfa305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T10:00:57.660 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 ceph-mds[65207]: -- 172.21.15.47:0/1995247678 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605a95e1860 con 0x5605a95bf400 2024-09-10T10:00:57.660 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k[65203]: starting mds.k at 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 -- 172.21.15.45:0/3162113064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca4101df0 msgr2=0x7f1ca41021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 --2- 172.21.15.45:0/3162113064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca4101df0 0x7f1ca41021d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009900 tx=0x7f1c9802a410 comp rx=0 tx=0).stop 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 -- 172.21.15.45:0/3162113064 shutdown_connections 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 --2- 172.21.15.45:0/3162113064 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca4106f20 0x7f1ca410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 --2- 172.21.15.45:0/3162113064 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ca41027a0 0x7f1ca41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 --2- 172.21.15.45:0/3162113064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca4101df0 0x7f1ca41021d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 -- 172.21.15.45:0/3162113064 >> 172.21.15.45:0/3162113064 conn(0x7f1ca40fd8f0 msgr2=0x7f1ca40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:57.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 -- 172.21.15.45:0/3162113064 shutdown_connections 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.845+0000 7f1caae9f640 1 -- 172.21.15.45:0/3162113064 wait complete. 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 Processor -- start 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 -- start start 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ca4101df0 0x7f1ca419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca41027a0 0x7f1ca419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca4106f20 0x7f1ca419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4111f80 con 0x7f1ca41027a0 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4111e00 con 0x7f1ca4101df0 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1caae9f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca4112100 con 0x7f1ca4106f20 2024-09-10T10:00:57.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca41027a0 0x7f1ca419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca41027a0 0x7f1ca419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44126/0 (socket says 172.21.15.45:44126) 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1ca3fff640 1 -- 172.21.15.45:0/2735953781 learned_addr learned my addr 172.21.15.45:0/2735953781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1ca8c3b640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ca4101df0 0x7f1ca419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.846+0000 7f1ca943c640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca4106f20 0x7f1ca419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca3fff640 1 -- 172.21.15.45:0/2735953781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca4106f20 msgr2=0x7f1ca419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca4106f20 0x7f1ca419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca3fff640 1 -- 172.21.15.45:0/2735953781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ca4101df0 msgr2=0x7f1ca419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ca4101df0 0x7f1ca419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca3fff640 1 -- 172.21.15.45:0/2735953781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca419f570 con 0x7f1ca41027a0 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca41027a0 0x7f1ca419ab00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000d950 tx=0x7f1c9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca1ffb640 1 -- 172.21.15.45:0/2735953781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90017070 con 0x7f1ca41027a0 2024-09-10T10:00:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca1ffb640 1 -- 172.21.15.45:0/2735953781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c90009c30 con 0x7f1ca41027a0 2024-09-10T10:00:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca419f860 con 0x7f1ca41027a0 2024-09-10T10:00:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1ca1ffb640 1 -- 172.21.15.45:0/2735953781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000f6b0 con 0x7f1ca41027a0 2024-09-10T10:00:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.847+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca41a7140 con 0x7f1ca41027a0 2024-09-10T10:00:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.849+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca41021d0 con 0x7f1ca41027a0 2024-09-10T10:00:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.849+0000 7f1ca1ffb640 1 -- 172.21.15.45:0/2735953781 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1c900109d0 con 0x7f1ca41027a0 2024-09-10T10:00:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.850+0000 7f1ca1ffb640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1c940a2b80 0x7f1c940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.850+0000 7f1ca8c3b640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1c940a2b80 0x7f1c940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.850+0000 7f1ca1ffb640 1 -- 172.21.15.45:0/2735953781 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f1c900ce830 con 0x7f1ca41027a0 2024-09-10T10:00:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.851+0000 7f1ca8c3b640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1c940a2b80 0x7f1c940a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009230 tx=0x7f1c980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:57.855+0000 7f1ca1ffb640 1 -- 172.21.15.45:0/2735953781 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c90009de0 con 0x7f1ca41027a0 2024-09-10T10:00:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.119+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ca41a73a0 con 0x7f1ca41027a0 2024-09-10T10:00:58.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.119+0000 7f1ca1ffb640 1 -- 172.21.15.45:0/2735953781 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f1c9009bc30 con 0x7f1ca41027a0 2024-09-10T10:00:58.122 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:00:58.122 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"btime":"2024-09-10T10:00:55:161295+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:00:58.122 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-09-10T10:00:58.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1c940a2b80 msgr2=0x7f1c940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1c940a2b80 0x7f1c940a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009230 tx=0x7f1c980058d0 comp rx=0 tx=0).stop 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca41027a0 msgr2=0x7f1ca419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca41027a0 0x7f1ca419ab00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000d950 tx=0x7f1c9000de10 comp rx=0 tx=0).stop 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 shutdown_connections 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1c940a2b80 0x7f1c940a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1ca4106f20 0x7f1ca419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ca41027a0 0x7f1ca419ab00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 --2- 172.21.15.45:0/2735953781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ca4101df0 0x7f1ca419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 >> 172.21.15.45:0/2735953781 conn(0x7f1ca40fd8f0 msgr2=0x7f1ca40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 shutdown_connections 2024-09-10T10:00:58.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:00:58.121+0000 7f1caae9f640 1 -- 172.21.15.45:0/2735953781 wait complete. 2024-09-10T10:00:58.159 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 bash[65188]: 1b558e67b92b6e7a5aa9e3da24299432c7c88ff4af97440e99cf665fbacfa305 2024-09-10T10:00:58.160 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:00:57 smithi047 systemd[1]: Started Ceph mds.k for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:00:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:58.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:58.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:58.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:58 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2735953781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:58.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:58.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:58 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2735953781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:58.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:00:58.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:00:58.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:00:58.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:00:58.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:58.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:58 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2735953781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:59.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:59 smithi045 ceph-mon[31114]: Deploying daemon mds.j on smithi045 2024-09-10T10:00:59.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:59 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.47:6832/2229149639,v1:172.21.15.47:6833/2229149639] up:boot 2024-09-10T10:00:59.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:59 smithi045 ceph-mon[31114]: fsmap 2 up:standby 2024-09-10T10:00:59.547 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:00:59 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:00:59.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:59 smithi162 ceph-mon[34963]: Deploying daemon mds.j on smithi045 2024-09-10T10:00:59.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:59 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.47:6832/2229149639,v1:172.21.15.47:6833/2229149639] up:boot 2024-09-10T10:00:59.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:59 smithi162 ceph-mon[34963]: fsmap 2 up:standby 2024-09-10T10:00:59.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:00:59 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:00:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:59 smithi047 ceph-mon[35244]: Deploying daemon mds.j on smithi045 2024-09-10T10:00:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:59 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.47:6832/2229149639,v1:172.21.15.47:6833/2229149639] up:boot 2024-09-10T10:00:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:59 smithi047 ceph-mon[35244]: fsmap 2 up:standby 2024-09-10T10:00:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:00:59 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:01:00.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:00 smithi045 ceph-mon[31114]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:00.401 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:01:00 smithi045 podman[71476]: 2024-09-10 10:01:00.343293254 +0000 UTC m=+0.147611276 container create bf77fe6945122a2f0c7029404d54a0a22a5733e58ac9a11daa2e5755f36392e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T10:01:00.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:00 smithi162 ceph-mon[34963]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:00.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:00 smithi047 ceph-mon[35244]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:00.900 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:01:00 smithi045 podman[71476]: 2024-09-10 10:01:00.579171246 +0000 UTC m=+0.383489266 container init bf77fe6945122a2f0c7029404d54a0a22a5733e58ac9a11daa2e5755f36392e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T10:01:00.900 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:01:00 smithi045 podman[71476]: 2024-09-10 10:01:00.582025643 +0000 UTC m=+0.386343670 container start bf77fe6945122a2f0c7029404d54a0a22a5733e58ac9a11daa2e5755f36392e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T10:01:00.900 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:01:00 smithi045 ceph-mds[71495]: -- 172.21.15.45:0/2202260288 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560570a23860 con 0x560570a01400 2024-09-10T10:01:00.901 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:01:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j[71491]: starting mds.j at 2024-09-10T10:01:00.901 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:01:00 smithi045 bash[71476]: bf77fe6945122a2f0c7029404d54a0a22a5733e58ac9a11daa2e5755f36392e5 2024-09-10T10:01:00.901 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:01:00 smithi045 systemd[1]: Started Ceph mds.j for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:01.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:01.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:01.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:01.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:01.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:01.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:01.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:01.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: Deploying daemon mds.d on smithi045 2024-09-10T10:01:01.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.45:6834/2062348864,v1:172.21.15.45:6835/2062348864] up:boot 2024-09-10T10:01:01.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: fsmap 3 up:standby 2024-09-10T10:01:01.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:01 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:01:02.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: Deploying daemon mds.d on smithi045 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.45:6834/2062348864,v1:172.21.15.45:6835/2062348864] up:boot 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: fsmap 3 up:standby 2024-09-10T10:01:02.062 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:01 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: Deploying daemon mds.d on smithi045 2024-09-10T10:01:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.45:6834/2062348864,v1:172.21.15.45:6835/2062348864] up:boot 2024-09-10T10:01:02.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: fsmap 3 up:standby 2024-09-10T10:01:02.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:01 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:01:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:02.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 -- 172.21.15.45:0/3748323590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 msgr2=0x7fd7641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 --2- 172.21.15.45:0/3748323590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd7641029b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd758009900 tx=0x7fd75802a480 comp rx=0 tx=0).stop 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 -- 172.21.15.45:0/3748323590 shutdown_connections 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 --2- 172.21.15.45:0/3748323590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd764102ef0 0x7fd76410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 --2- 172.21.15.45:0/3748323590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd7641029b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 --2- 172.21.15.45:0/3748323590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd764100e20 0x7fd764101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 -- 172.21.15.45:0/3748323590 >> 172.21.15.45:0/3748323590 conn(0x7fd7640fc7d0 msgr2=0x7fd7640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 -- 172.21.15.45:0/3748323590 shutdown_connections 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.841+0000 7fd76a01f640 1 -- 172.21.15.45:0/3748323590 wait complete. 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 Processor -- start 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 -- start start 2024-09-10T10:01:02.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd764100e20 0x7fd76419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd76419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd764102ef0 0x7fd76419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd764110fa0 con 0x7fd764102570 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd764110e20 con 0x7fd764102ef0 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd76a01f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd764111120 con 0x7fd764100e20 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.842+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd76419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd76419abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44154/0 (socket says 172.21.15.45:44154) 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 -- 172.21.15.45:0/718803832 learned_addr learned my addr 172.21.15.45:0/718803832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd763fff640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd764102ef0 0x7fd76419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd7637fe640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd764100e20 0x7fd76419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 -- 172.21.15.45:0/718803832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd764100e20 msgr2=0x7fd76419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd764100e20 0x7fd76419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:02.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 -- 172.21.15.45:0/718803832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd764102ef0 msgr2=0x7fd76419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd764102ef0 0x7fd76419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 -- 172.21.15.45:0/718803832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76419f650 con 0x7fd764102570 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd7637fe640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd764100e20 0x7fd76419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd762ffd640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd76419abe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd758006010 tx=0x7fd758003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd760ff9640 1 -- 172.21.15.45:0/718803832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75803f070 con 0x7fd764102570 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76419f8e0 con 0x7fd764102570 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd760ff9640 1 -- 172.21.15.45:0/718803832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd758031e30 con 0x7fd764102570 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd760ff9640 1 -- 172.21.15.45:0/718803832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd758035030 con 0x7fd764102570 2024-09-10T10:01:02.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.843+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7641a7220 con 0x7fd764102570 2024-09-10T10:01:02.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.845+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd764101200 con 0x7fd764102570 2024-09-10T10:01:02.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.846+0000 7fd760ff9640 1 -- 172.21.15.45:0/718803832 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd758042070 con 0x7fd764102570 2024-09-10T10:01:02.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.846+0000 7fd760ff9640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd74c0a2b80 0x7fd74c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:02.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.846+0000 7fd7637fe640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd74c0a2b80 0x7fd74c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:02.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.846+0000 7fd760ff9640 1 -- 172.21.15.45:0/718803832 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd7580ef590 con 0x7fd764102570 2024-09-10T10:01:02.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.847+0000 7fd7637fe640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd74c0a2b80 0x7fd74c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd75400abe0 tx=0x7fd75400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:02.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:02.850+0000 7fd760ff9640 1 -- 172.21.15.45:0/718803832 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7580bc8f0 con 0x7fd764102570 2024-09-10T10:01:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.066+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7641a7450 con 0x7fd764102570 2024-09-10T10:01:03.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.067+0000 7fd760ff9640 1 -- 172.21.15.45:0/718803832 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fd7580bc1f0 con 0x7fd764102570 2024-09-10T10:01:03.068 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:03.069 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":4,"btime":"2024-09-10T10:01:01:340656+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T10:01:03.069 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 4 2024-09-10T10:01:03.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd74c0a2b80 msgr2=0x7fd74c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd74c0a2b80 0x7fd74c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd75400abe0 tx=0x7fd75400a5c0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 msgr2=0x7fd76419abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd76419abe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd758006010 tx=0x7fd758003c80 comp rx=0 tx=0).stop 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 shutdown_connections 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd74c0a2b80 0x7fd74c0a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd764102ef0 0x7fd76419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd764102570 0x7fd76419abe0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 --2- 172.21.15.45:0/718803832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd764100e20 0x7fd76419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 >> 172.21.15.45:0/718803832 conn(0x7fd7640fc7d0 msgr2=0x7fd764103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 shutdown_connections 2024-09-10T10:01:03.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:03.069+0000 7fd76a01f640 1 -- 172.21.15.45:0/718803832 wait complete. 2024-09-10T10:01:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:03 smithi162 ceph-mon[34963]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:03 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/718803832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:04.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:03 smithi045 ceph-mon[31114]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:04.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:03 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/718803832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:04.150 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 10 10:01:03 smithi045 systemd[1]: Started Ceph mds.d for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:04.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:03 smithi047 ceph-mon[35244]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:04.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:03 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/718803832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:05.086 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: Deploying daemon mds.e on smithi047 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.45:6836/1314634033,v1:172.21.15.45:6837/1314634033] up:boot 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: fsmap 4 up:standby 2024-09-10T10:01:05.087 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:04 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:01:05.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: Deploying daemon mds.e on smithi047 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.45:6836/1314634033,v1:172.21.15.45:6837/1314634033] up:boot 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: fsmap 4 up:standby 2024-09-10T10:01:05.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:04 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: Deploying daemon mds.e on smithi047 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.45:6836/1314634033,v1:172.21.15.45:6837/1314634033] up:boot 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: fsmap 4 up:standby 2024-09-10T10:01:05.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:04 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:01:05.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:05.910 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:05 smithi047 systemd[1]: Starting Ceph mds.e for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:01:06.005 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:06.127 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:05 smithi045 ceph-mon[31114]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:06.266 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:06 smithi047 ceph-mon[35244]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:06.267 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 podman[65518]: 2024-09-10 10:01:05.917927832 +0000 UTC m=+0.016407322 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:01:06.267 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 podman[65518]: 2024-09-10 10:01:06.024532249 +0000 UTC m=+0.123011780 container create 0ca7f833c8d3ac44d6346d66270300dae12f5d0e1c651edaf670af3b8d53aaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T10:01:06.267 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 podman[65518]: 2024-09-10 10:01:06.240406696 +0000 UTC m=+0.338886188 container init 0ca7f833c8d3ac44d6346d66270300dae12f5d0e1c651edaf670af3b8d53aaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:01:06.267 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 podman[65518]: 2024-09-10 10:01:06.243129049 +0000 UTC m=+0.341608539 container start 0ca7f833c8d3ac44d6346d66270300dae12f5d0e1c651edaf670af3b8d53aaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T10:01:06.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:06 smithi162 ceph-mon[34963]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 -- 172.21.15.45:0/3858896368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100480 msgr2=0x7f065c100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 --2- 172.21.15.45:0/3858896368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100480 0x7f065c100860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f064c009900 tx=0x7f064c02a410 comp rx=0 tx=0).stop 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 -- 172.21.15.45:0/3858896368 shutdown_connections 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 --2- 172.21.15.45:0/3858896368 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f065c1055b0 0x7f065c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 --2- 172.21.15.45:0/3858896368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f065c100e30 0x7f065c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 --2- 172.21.15.45:0/3858896368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100480 0x7f065c100860 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 -- 172.21.15.45:0/3858896368 >> 172.21.15.45:0/3858896368 conn(0x7f065c077210 msgr2=0x7f065c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 -- 172.21.15.45:0/3858896368 shutdown_connections 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.505+0000 7f06612e1640 1 -- 172.21.15.45:0/3858896368 wait complete. 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 Processor -- start 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 -- start start 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f065c100480 0x7f065c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100e30 0x7f065c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f065c1055b0 0x7f065c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10fde0 con 0x7f065c100e30 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10fc60 con 0x7f065c1055b0 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.506+0000 7f06612e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10ff60 con 0x7f065c100480 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100e30 0x7f065c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f065affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f065c100480 0x7f065c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100e30 0x7f065c19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58414/0 (socket says 172.21.15.45:58414) 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 -- 172.21.15.45:0/794994910 learned_addr learned my addr 172.21.15.45:0/794994910 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f065b7fe640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f065c1055b0 0x7f065c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 -- 172.21.15.45:0/794994910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f065c100480 msgr2=0x7f065c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f065c100480 0x7f065c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 -- 172.21.15.45:0/794994910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f065c1055b0 msgr2=0x7f065c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f065c1055b0 0x7f065c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 -- 172.21.15.45:0/794994910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c19f580 con 0x7f065c100e30 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0653fff640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100e30 0x7f065c19ab10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f064400c920 tx=0x7f064400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0658ff9640 1 -- 172.21.15.45:0/794994910 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644007c70 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c19f870 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0658ff9640 1 -- 172.21.15.45:0/794994910 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0644007e10 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f0658ff9640 1 -- 172.21.15.45:0/794994910 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644018890 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.507+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1a7150 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.508+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c100860 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.510+0000 7f0658ff9640 1 -- 172.21.15.45:0/794994910 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0644018a70 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.511+0000 7f0658ff9640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f06480a2b80 0x7f06480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.511+0000 7f065affd640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f06480a2b80 0x7f06480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.511+0000 7f0658ff9640 1 -- 172.21.15.45:0/794994910 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f06440cec40 con 0x7f065c100e30 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.511+0000 7f065affd640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f06480a2b80 0x7f06480a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f064c000c00 tx=0x7f064c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.515+0000 7f0658ff9640 1 -- 172.21.15.45:0/794994910 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0644014030 con 0x7f065c100e30 2024-09-10T10:01:06.659 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 ceph-mds[65537]: -- 172.21.15.47:0/4026816260 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56214d0ff860 con 0x56214d0dd000 2024-09-10T10:01:06.660 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e[65533]: starting mds.e at 2024-09-10T10:01:06.660 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 bash[65518]: 0ca7f833c8d3ac44d6346d66270300dae12f5d0e1c651edaf670af3b8d53aaf1 2024-09-10T10:01:06.660 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:01:06 smithi047 systemd[1]: Started Ceph mds.e for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.730+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f065c02d7c0 con 0x7f065c100e30 2024-09-10T10:01:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.731+0000 7f0658ff9640 1 -- 172.21.15.45:0/794994910 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3910 (secure 0 0 0) 0x7f064409c040 con 0x7f065c100e30 2024-09-10T10:01:06.732 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:06.733 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"btime":"2024-09-10T10:01:04:731979+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T10:01:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f06480a2b80 msgr2=0x7f06480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f06480a2b80 0x7f06480a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f064c000c00 tx=0x7f064c005b50 comp rx=0 tx=0).stop 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100e30 msgr2=0x7f065c19ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100e30 0x7f065c19ab10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f064400c920 tx=0x7f064400cde0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 shutdown_connections 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f06480a2b80 0x7f06480a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f065c1055b0 0x7f065c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f065c100e30 0x7f065c19ab10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 --2- 172.21.15.45:0/794994910 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f065c100480 0x7f065c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 >> 172.21.15.45:0/794994910 conn(0x7f065c077210 msgr2=0x7f065c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 shutdown_connections 2024-09-10T10:01:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:06.733+0000 7f06612e1640 1 -- 172.21.15.45:0/794994910 wait complete. 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: Deploying daemon mds.f on smithi162 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/794994910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.47:6834/2176252394,v1:172.21.15.47:6835/2176252394] up:boot 2024-09-10T10:01:07.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: fsmap 5 up:standby 2024-09-10T10:01:07.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:07 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:01:07.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:07.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:07.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:07.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: Deploying daemon mds.f on smithi162 2024-09-10T10:01:07.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/794994910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:07.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.47:6834/2176252394,v1:172.21.15.47:6835/2176252394] up:boot 2024-09-10T10:01:07.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: fsmap 5 up:standby 2024-09-10T10:01:07.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:07 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:01:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: Deploying daemon mds.f on smithi162 2024-09-10T10:01:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/794994910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.47:6834/2176252394,v1:172.21.15.47:6835/2176252394] up:boot 2024-09-10T10:01:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: fsmap 5 up:standby 2024-09-10T10:01:07.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:07 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:01:08.400 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 systemd[1]: Starting Ceph mds.f for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:01:08.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:08 smithi047 ceph-mon[35244]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:08.682 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:08 smithi162 ceph-mon[34963]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:08.682 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 podman[64127]: 2024-09-10 10:01:08.329520097 +0000 UTC m=+0.015699878 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:01:08.682 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 podman[64127]: 2024-09-10 10:01:08.429201602 +0000 UTC m=+0.115381367 container create 55f6f8d1ef32dfd575112bb8fc857ee7fc8f0173c13ba9c22474b24a7173045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T10:01:08.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:08 smithi045 ceph-mon[31114]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:09.149 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 podman[64127]: 2024-09-10 10:01:08.681514238 +0000 UTC m=+0.367694005 container init 55f6f8d1ef32dfd575112bb8fc857ee7fc8f0173c13ba9c22474b24a7173045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:01:09.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 podman[64127]: 2024-09-10 10:01:08.684619074 +0000 UTC m=+0.370798845 container start 55f6f8d1ef32dfd575112bb8fc857ee7fc8f0173c13ba9c22474b24a7173045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:01:09.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 ceph-mds[64146]: -- 172.21.15.162:0/82645272 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8754f3860 con 0x55e8754d0800 2024-09-10T10:01:09.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f[64142]: starting mds.f at 2024-09-10T10:01:09.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 bash[64127]: 55f6f8d1ef32dfd575112bb8fc857ee7fc8f0173c13ba9c22474b24a7173045b 2024-09-10T10:01:09.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:01:08 smithi162 systemd[1]: Started Ceph mds.f for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:09.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:09.901 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:10.022 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: Deploying daemon mds.c on smithi162 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.162:6834/1763949384,v1:172.21.15.162:6835/1763949384] up:boot 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: fsmap 6 up:standby 2024-09-10T10:01:10.023 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:09 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:01:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: Deploying daemon mds.c on smithi162 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.162:6834/1763949384,v1:172.21.15.162:6835/1763949384] up:boot 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: fsmap 6 up:standby 2024-09-10T10:01:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:09 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: Deploying daemon mds.c on smithi162 2024-09-10T10:01:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.162:6834/1763949384,v1:172.21.15.162:6835/1763949384] up:boot 2024-09-10T10:01:10.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: fsmap 6 up:standby 2024-09-10T10:01:10.161 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:09 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:01:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 -- 172.21.15.45:0/1073827324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94103780 msgr2=0x7f6a94107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1073827324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94103780 0x7f6a94107bc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009930 tx=0x7f6a8802a420 comp rx=0 tx=0).stop 2024-09-10T10:01:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 -- 172.21.15.45:0/1073827324 shutdown_connections 2024-09-10T10:01:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1073827324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a94108300 0x7f6a9410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1073827324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94103780 0x7f6a94107bc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1073827324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a94102dd0 0x7f6a941031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 -- 172.21.15.45:0/1073827324 >> 172.21.15.45:0/1073827324 conn(0x7f6a940fc7d0 msgr2=0x7f6a940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 -- 172.21.15.45:0/1073827324 shutdown_connections 2024-09-10T10:01:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 -- 172.21.15.45:0/1073827324 wait complete. 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.404+0000 7f6a9a285640 1 Processor -- start 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a9a285640 1 -- start start 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a9a285640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94102dd0 0x7f6a941985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a9a285640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a94103780 0x7f6a94198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a9a285640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a94108300 0x7f6a9419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a9a285640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94112070 con 0x7f6a94102dd0 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a9a285640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94111ef0 con 0x7f6a94103780 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a9a285640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6a941121f0 con 0x7f6a94108300 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94102dd0 0x7f6a941985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94102dd0 0x7f6a941985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58434/0 (socket says 172.21.15.45:58434) 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a93fff640 1 -- 172.21.15.45:0/1811645303 learned_addr learned my addr 172.21.15.45:0/1811645303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a98822640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a94108300 0x7f6a9419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.405+0000 7f6a937fe640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a94103780 0x7f6a94198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a93fff640 1 -- 172.21.15.45:0/1811645303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a94108300 msgr2=0x7f6a9419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a93fff640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a94108300 0x7f6a9419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a93fff640 1 -- 172.21.15.45:0/1811645303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a94103780 msgr2=0x7f6a94198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a93fff640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a94103780 0x7f6a94198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a93fff640 1 -- 172.21.15.45:0/1811645303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9419d5a0 con 0x7f6a94102dd0 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a93fff640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94102dd0 0x7f6a941985f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400d770 tx=0x7f6a8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a917fa640 1 -- 172.21.15.45:0/1811645303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84017070 con 0x7f6a94102dd0 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a917fa640 1 -- 172.21.15.45:0/1811645303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a84005500 con 0x7f6a94102dd0 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9419d890 con 0x7f6a94102dd0 2024-09-10T10:01:10.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a917fa640 1 -- 172.21.15.45:0/1811645303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84004e00 con 0x7f6a94102dd0 2024-09-10T10:01:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.406+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941a5170 con 0x7f6a94102dd0 2024-09-10T10:01:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.407+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a941031b0 con 0x7f6a94102dd0 2024-09-10T10:01:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.408+0000 7f6a917fa640 1 -- 172.21.15.45:0/1811645303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6a84004fa0 con 0x7f6a94102dd0 2024-09-10T10:01:10.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.409+0000 7f6a917fa640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a7c0a2b80 0x7f6a7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.409+0000 7f6a937fe640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a7c0a2b80 0x7f6a7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.409+0000 7f6a917fa640 1 -- 172.21.15.45:0/1811645303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f6a840ce570 con 0x7f6a94102dd0 2024-09-10T10:01:10.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.409+0000 7f6a937fe640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a7c0a2b80 0x7f6a7c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6a88005940 tx=0x7f6a880058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.413+0000 7f6a917fa640 1 -- 172.21.15.45:0/1811645303 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a8409b320 con 0x7f6a94102dd0 2024-09-10T10:01:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.626+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a9410a290 con 0x7f6a94102dd0 2024-09-10T10:01:10.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.626+0000 7f6a917fa640 1 -- 172.21.15.45:0/1811645303 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7f6a8409f620 con 0x7f6a94102dd0 2024-09-10T10:01:10.628 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:10.628 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"btime":"2024-09-10T10:01:09:402740+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-10T10:01:10.628 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.629+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a7c0a2b80 msgr2=0x7f6a7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.629+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a7c0a2b80 0x7f6a7c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6a88005940 tx=0x7f6a880058d0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.629+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94102dd0 msgr2=0x7f6a941985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.629+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94102dd0 0x7f6a941985f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400d770 tx=0x7f6a8400dc30 comp rx=0 tx=0).stop 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.629+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 shutdown_connections 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.629+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a7c0a2b80 0x7f6a7c0a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.629+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a94108300 0x7f6a9419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.630+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a94103780 0x7f6a94198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.630+0000 7f6a9a285640 1 --2- 172.21.15.45:0/1811645303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a94102dd0 0x7f6a941985f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.630+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 >> 172.21.15.45:0/1811645303 conn(0x7f6a940fc7d0 msgr2=0x7f6a94105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.630+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 shutdown_connections 2024-09-10T10:01:10.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:10.630+0000 7f6a9a285640 1 -- 172.21.15.45:0/1811645303 wait complete. 2024-09-10T10:01:10.833 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:10 smithi162 systemd[1]: Starting Ceph mds.c for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:01:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:10 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1811645303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:11.093 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:10 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1811645303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:11.094 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:10 smithi162 podman[64426]: 2024-09-10 10:01:10.790076131 +0000 UTC m=+0.016199885 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:01:11.094 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:10 smithi162 podman[64426]: 2024-09-10 10:01:10.890414826 +0000 UTC m=+0.116538611 container create b1281eee2588ee97af4b46cafc47f7a77727dccd5fc9dd8c85039b513d2e9dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T10:01:11.094 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:11 smithi162 podman[64426]: 2024-09-10 10:01:11.067445489 +0000 UTC m=+0.293569232 container init b1281eee2588ee97af4b46cafc47f7a77727dccd5fc9dd8c85039b513d2e9dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:01:11.094 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:11 smithi162 podman[64426]: 2024-09-10 10:01:11.070117692 +0000 UTC m=+0.296241446 container start b1281eee2588ee97af4b46cafc47f7a77727dccd5fc9dd8c85039b513d2e9dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T10:01:11.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:10 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1811645303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:11.399 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:11 smithi162 ceph-mds[64445]: -- 172.21.15.162:0/1118665239 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fafed31860 con 0x55fafed0f000 2024-09-10T10:01:11.399 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c[64441]: starting mds.c at 2024-09-10T10:01:11.400 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:11 smithi162 bash[64426]: b1281eee2588ee97af4b46cafc47f7a77727dccd5fc9dd8c85039b513d2e9dbf 2024-09-10T10:01:11.400 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:01:11 smithi162 systemd[1]: Started Ceph mds.c for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:12.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: Deploying daemon mds.h on smithi047 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.162:6836/3408858695,v1:172.21.15.162:6837/3408858695] up:boot 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: fsmap 7 up:standby 2024-09-10T10:01:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:12 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:01:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: Deploying daemon mds.h on smithi047 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.162:6836/3408858695,v1:172.21.15.162:6837/3408858695] up:boot 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: fsmap 7 up:standby 2024-09-10T10:01:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:12 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: Deploying daemon mds.h on smithi047 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.162:6836/3408858695,v1:172.21.15.162:6837/3408858695] up:boot 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: fsmap 7 up:standby 2024-09-10T10:01:12.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:12 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:01:13.307 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:13.331 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 systemd[1]: Starting Ceph mds.h for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:01:13.576 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:13.636 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 podman[65822]: 2024-09-10 10:01:13.23304634 +0000 UTC m=+0.016219876 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:01:13.636 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 podman[65822]: 2024-09-10 10:01:13.341730523 +0000 UTC m=+0.124904059 container create 5548595c5848fd007ae326a8a25ff42a62b941142530631296e3fe9130d31965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T10:01:13.636 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 podman[65822]: 2024-09-10 10:01:13.535254294 +0000 UTC m=+0.318427831 container init 5548595c5848fd007ae326a8a25ff42a62b941142530631296e3fe9130d31965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T10:01:13.636 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 podman[65822]: 2024-09-10 10:01:13.537928508 +0000 UTC m=+0.321102053 container start 5548595c5848fd007ae326a8a25ff42a62b941142530631296e3fe9130d31965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T10:01:13.637 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 ceph-mds[65842]: -- 172.21.15.47:0/899776836 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e26d63860 con 0x558e26d41400 2024-09-10T10:01:13.637 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h[65838]: starting mds.h at 2024-09-10T10:01:13.637 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 bash[65822]: 5548595c5848fd007ae326a8a25ff42a62b941142530631296e3fe9130d31965 2024-09-10T10:01:13.910 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:01:13 smithi047 systemd[1]: Started Ceph mds.h for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:14.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 -- 172.21.15.45:0/2947126366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c106f10 msgr2=0x7fca6c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:14.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2947126366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c106f10 0x7fca6c10ea90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fca64009ca0 tx=0x7fca6402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 -- 172.21.15.45:0/2947126366 shutdown_connections 2024-09-10T10:01:14.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2947126366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c106f10 0x7fca6c10ea90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2947126366 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca6c102790 0x7fca6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2947126366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c101de0 0x7fca6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 -- 172.21.15.45:0/2947126366 >> 172.21.15.45:0/2947126366 conn(0x7fca6c0fd8c0 msgr2=0x7fca6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 -- 172.21.15.45:0/2947126366 shutdown_connections 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.020+0000 7fca72cb5640 1 -- 172.21.15.45:0/2947126366 wait complete. 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 Processor -- start 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 -- start start 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c101de0 0x7fca6c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca6c102790 0x7fca6c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c106f10 0x7fca6c10dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c111f70 con 0x7fca6c101de0 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c111df0 con 0x7fca6c106f10 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca72cb5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1120f0 con 0x7fca6c102790 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca70a51640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c101de0 0x7fca6c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:14.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca70a51640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c101de0 0x7fca6c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42960/0 (socket says 172.21.15.45:42960) 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca70a51640 1 -- 172.21.15.45:0/2236969650 learned_addr learned my addr 172.21.15.45:0/2236969650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca6bfff640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca6c102790 0x7fca6c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.021+0000 7fca71252640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c106f10 0x7fca6c10dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca70a51640 1 -- 172.21.15.45:0/2236969650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca6c102790 msgr2=0x7fca6c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca70a51640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca6c102790 0x7fca6c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca70a51640 1 -- 172.21.15.45:0/2236969650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c106f10 msgr2=0x7fca6c10dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca70a51640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c106f10 0x7fca6c10dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca70a51640 1 -- 172.21.15.45:0/2236969650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c10e520 con 0x7fca6c101de0 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca6bfff640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca6c102790 0x7fca6c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca70a51640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c101de0 0x7fca6c19f000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca5400c890 tx=0x7fca5400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca69ffb640 1 -- 172.21.15.45:0/2236969650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca54004220 con 0x7fca6c101de0 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca69ffb640 1 -- 172.21.15.45:0/2236969650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca540043c0 con 0x7fca6c101de0 2024-09-10T10:01:14.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c10e7b0 con 0x7fca6c101de0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca69ffb640 1 -- 172.21.15.45:0/2236969650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca540049e0 con 0x7fca6c101de0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.022+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c19ac20 con 0x7fca6c101de0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.023+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c1032c0 con 0x7fca6c101de0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.024+0000 7fca69ffb640 1 -- 172.21.15.45:0/2236969650 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca540079b0 con 0x7fca6c101de0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.025+0000 7fca69ffb640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fca580a2b80 0x7fca580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.025+0000 7fca6bfff640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fca580a2b80 0x7fca580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.025+0000 7fca69ffb640 1 -- 172.21.15.45:0/2236969650 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fca540ceb70 con 0x7fca6c101de0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.025+0000 7fca6bfff640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fca580a2b80 0x7fca580a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fca6c197780 tx=0x7fca5c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:14.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.029+0000 7fca69ffb640 1 -- 172.21.15.45:0/2236969650 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca54014030 con 0x7fca6c101de0 2024-09-10T10:01:14.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.243+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca6c19aee0 con 0x7fca6c101de0 2024-09-10T10:01:14.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.243+0000 7fca69ffb640 1 -- 172.21.15.45:0/2236969650 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7fca5409b920 con 0x7fca6c101de0 2024-09-10T10:01:14.245 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:14.245 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"btime":"2024-09-10T10:01:14:176449+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T10:01:14.245 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fca580a2b80 msgr2=0x7fca580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fca580a2b80 0x7fca580a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fca6c197780 tx=0x7fca5c00b390 comp rx=0 tx=0).stop 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c101de0 msgr2=0x7fca6c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c101de0 0x7fca6c19f000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca5400c890 tx=0x7fca5400cd50 comp rx=0 tx=0).stop 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 shutdown_connections 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fca580a2b80 0x7fca580a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca6c106f10 0x7fca6c10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca6c102790 0x7fca6c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 --2- 172.21.15.45:0/2236969650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca6c101de0 0x7fca6c19f000 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.245+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 >> 172.21.15.45:0/2236969650 conn(0x7fca6c0fd8c0 msgr2=0x7fca6c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.246+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 shutdown_connections 2024-09-10T10:01:14.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:14.246+0000 7fca72cb5640 1 -- 172.21.15.45:0/2236969650 wait complete. 2024-09-10T10:01:14.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:14 smithi045 ceph-mon[31114]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:14.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.294 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:14.295 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:14.295 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:14 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:14.363 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:14 smithi162 ceph-mon[34963]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:14.417 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:14 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.417 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:14 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.417 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:14 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.417 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:14 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:14.417 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:14 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:14.417 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:14 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:14.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:14 smithi047 ceph-mon[35244]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:14.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:14 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:14 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:14 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:14.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:14 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:14.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:14 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:14.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:14 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:15.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:15 smithi162 ceph-mon[34963]: Deploying daemon mds.i on smithi162 2024-09-10T10:01:15.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:15 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.47:6836/2773219506,v1:172.21.15.47:6837/2773219506] up:boot 2024-09-10T10:01:15.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:15 smithi162 ceph-mon[34963]: fsmap 8 up:standby 2024-09-10T10:01:15.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:15 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:01:15.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:15 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2236969650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:15 smithi045 ceph-mon[31114]: Deploying daemon mds.i on smithi162 2024-09-10T10:01:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:15 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.47:6836/2773219506,v1:172.21.15.47:6837/2773219506] up:boot 2024-09-10T10:01:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:15 smithi045 ceph-mon[31114]: fsmap 8 up:standby 2024-09-10T10:01:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:15 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:01:15.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:15 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2236969650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:15.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:15 smithi047 ceph-mon[35244]: Deploying daemon mds.i on smithi162 2024-09-10T10:01:15.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:15 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.47:6836/2773219506,v1:172.21.15.47:6837/2773219506] up:boot 2024-09-10T10:01:15.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:15 smithi047 ceph-mon[35244]: fsmap 8 up:standby 2024-09-10T10:01:15.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:15 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:01:15.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:15 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2236969650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:16 smithi162 ceph-mon[34963]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:16.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:16 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:16.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:16 smithi045 ceph-mon[31114]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:16.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:16.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:16.563 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:16 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:16.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:16 smithi047 ceph-mon[35244]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:16.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:16.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:16.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:16.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:16 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:16.941 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:17.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:17 smithi045 ceph-mon[31114]: Deploying daemon mds.g on smithi045 2024-09-10T10:01:17.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:17 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.162:6838/2065513838,v1:172.21.15.162:6839/2065513838] up:boot 2024-09-10T10:01:17.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:17 smithi045 ceph-mon[31114]: fsmap 9 up:standby 2024-09-10T10:01:17.462 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:17 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:01:17.527 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:17.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:17 smithi162 ceph-mon[34963]: Deploying daemon mds.g on smithi045 2024-09-10T10:01:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:17 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.162:6838/2065513838,v1:172.21.15.162:6839/2065513838] up:boot 2024-09-10T10:01:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:17 smithi162 ceph-mon[34963]: fsmap 9 up:standby 2024-09-10T10:01:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:17 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:01:17.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:17 smithi047 ceph-mon[35244]: Deploying daemon mds.g on smithi045 2024-09-10T10:01:17.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:17 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.162:6838/2065513838,v1:172.21.15.162:6839/2065513838] up:boot 2024-09-10T10:01:17.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:17 smithi047 ceph-mon[35244]: fsmap 9 up:standby 2024-09-10T10:01:17.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:17 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:01:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.165+0000 7f05f33bc640 1 -- 172.21.15.45:0/11849114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 msgr2=0x7f05ec110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.165+0000 7f05f33bc640 1 --2- 172.21.15.45:0/11849114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec110830 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009ca0 tx=0x7f05e402a950 comp rx=0 tx=0).stop 2024-09-10T10:01:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.165+0000 7f05f33bc640 1 -- 172.21.15.45:0/11849114 shutdown_connections 2024-09-10T10:01:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.165+0000 7f05f33bc640 1 --2- 172.21.15.45:0/11849114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec110830 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.165+0000 7f05f33bc640 1 --2- 172.21.15.45:0/11849114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05ec074e30 0x7f05ec10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.165+0000 7f05f33bc640 1 --2- 172.21.15.45:0/11849114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05ec074510 0x7f05ec0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.165+0000 7f05f33bc640 1 -- 172.21.15.45:0/11849114 >> 172.21.15.45:0/11849114 conn(0x7f05ec0fd8c0 msgr2=0x7f05ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 -- 172.21.15.45:0/11849114 shutdown_connections 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 -- 172.21.15.45:0/11849114 wait complete. 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 Processor -- start 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 -- start start 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05ec074510 0x7f05ec19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05ec074e30 0x7f05ec19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec114150 con 0x7f05ec10e440 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec113fd0 con 0x7f05ec074510 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.166+0000 7f05f33bc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1142d0 con 0x7f05ec074e30 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42978/0 (socket says 172.21.15.45:42978) 2024-09-10T10:01:18.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 -- 172.21.15.45:0/1672517155 learned_addr learned my addr 172.21.15.45:0/1672517155 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1158640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05ec074510 0x7f05ec19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f0957640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05ec074e30 0x7f05ec19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 -- 172.21.15.45:0/1672517155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05ec074e30 msgr2=0x7f05ec19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05ec074e30 0x7f05ec19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 -- 172.21.15.45:0/1672517155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05ec074510 msgr2=0x7f05ec19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05ec074510 0x7f05ec19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 -- 172.21.15.45:0/1672517155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec19f570 con 0x7f05ec10e440 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1158640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05ec074510 0x7f05ec19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f1959640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec19ee90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05e402ae70 tx=0x7f05e402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05e27fc640 1 -- 172.21.15.45:0/1672517155 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4034040 con 0x7f05ec10e440 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec19f800 con 0x7f05ec10e440 2024-09-10T10:01:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05e27fc640 1 -- 172.21.15.45:0/1672517155 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e4030920 con 0x7f05ec10e440 2024-09-10T10:01:18.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05e27fc640 1 -- 172.21.15.45:0/1672517155 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4045480 con 0x7f05ec10e440 2024-09-10T10:01:18.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.167+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1a7140 con 0x7f05ec10e440 2024-09-10T10:01:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.169+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec0748f0 con 0x7f05ec10e440 2024-09-10T10:01:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.170+0000 7f05e27fc640 1 -- 172.21.15.45:0/1672517155 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f05e4037020 con 0x7f05ec10e440 2024-09-10T10:01:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.170+0000 7f05e27fc640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f05d80a2b80 0x7f05d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.170+0000 7f05f1158640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f05d80a2b80 0x7f05d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.170+0000 7f05e27fc640 1 -- 172.21.15.45:0/1672517155 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f05e40efb50 con 0x7f05ec10e440 2024-09-10T10:01:18.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.171+0000 7f05f1158640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f05d80a2b80 0x7f05d80a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05e800abe0 tx=0x7f05e800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:18.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.175+0000 7f05e27fc640 1 -- 172.21.15.45:0/1672517155 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05e40bcf50 con 0x7f05ec10e440 2024-09-10T10:01:18.381 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:18 smithi045 ceph-mon[31114]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.395+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05ec02d7c0 con 0x7f05ec10e440 2024-09-10T10:01:18.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.395+0000 7f05e27fc640 1 -- 172.21.15.45:0/1672517155 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8046 (secure 0 0 0) 0x7f05e4043e40 con 0x7f05ec10e440 2024-09-10T10:01:18.397 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:18.397 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"btime":"2024-09-10T10:01:16:185879+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:01:18.397 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.397+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f05d80a2b80 msgr2=0x7f05d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f05d80a2b80 0x7f05d80a5040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f05e800abe0 tx=0x7f05e800a5c0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 msgr2=0x7f05ec19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec19ee90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f05e402ae70 tx=0x7f05e402f5b0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 shutdown_connections 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f05d80a2b80 0x7f05d80a5040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05ec10e440 0x7f05ec19ee90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f05ec074e30 0x7f05ec19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 --2- 172.21.15.45:0/1672517155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05ec074510 0x7f05ec19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:18.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 >> 172.21.15.45:0/1672517155 conn(0x7f05ec0fd8c0 msgr2=0x7f05ec0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 shutdown_connections 2024-09-10T10:01:18.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:18.398+0000 7f05f33bc640 1 -- 172.21.15.45:0/1672517155 wait complete. 2024-09-10T10:01:18.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:18 smithi162 ceph-mon[34963]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:18.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:18 smithi047 ceph-mon[35244]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:19.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:19 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1672517155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:19.392 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:01:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-g[72690]: starting mds.g at 2024-09-10T10:01:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:19 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1672517155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:19.650 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:01:19 smithi045 bash[72662]: f0e788d91c92f4e0753dee441844b85db49b2d7561bb68dce0c2011163c1830b 2024-09-10T10:01:19.650 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:01:19 smithi045 systemd[1]: Started Ceph mds.g for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:19.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:19 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1672517155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:20.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: Deploying daemon mds.a on smithi045 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.45:6838/743802171,v1:172.21.15.45:6839/743802171] up:boot 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: fsmap 10 up:standby 2024-09-10T10:01:20.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:20 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: Deploying daemon mds.a on smithi045 2024-09-10T10:01:20.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.45:6838/743802171,v1:172.21.15.45:6839/743802171] up:boot 2024-09-10T10:01:20.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: fsmap 10 up:standby 2024-09-10T10:01:20.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:20 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: Deploying daemon mds.a on smithi045 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.45:6838/743802171,v1:172.21.15.45:6839/743802171] up:boot 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: fsmap 10 up:standby 2024-09-10T10:01:20.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:20 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:01:21.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:21.400 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 systemd[1]: Starting Ceph mds.a for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:01:21.781 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 podman[72995]: 2024-09-10 10:01:21.403038994 +0000 UTC m=+0.016676590 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:01:21.781 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 podman[72995]: 2024-09-10 10:01:21.526970253 +0000 UTC m=+0.140607849 container create ff9749f130a3f8107d93e6345925616f2d445397076c1c7f6b056e44aa1c99a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T10:01:21.781 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 podman[72995]: 2024-09-10 10:01:21.779959787 +0000 UTC m=+0.393597383 container init ff9749f130a3f8107d93e6345925616f2d445397076c1c7f6b056e44aa1c99a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T10:01:21.905 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:22.031 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 podman[72995]: 2024-09-10 10:01:21.7831756 +0000 UTC m=+0.396813196 container start ff9749f130a3f8107d93e6345925616f2d445397076c1c7f6b056e44aa1c99a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T10:01:22.031 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 ceph-mds[73039]: -- 172.21.15.45:0/3651336099 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f08023860 con 0x564f08000800 2024-09-10T10:01:22.031 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a[73022]: starting mds.a at 2024-09-10T10:01:22.031 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 bash[72995]: ff9749f130a3f8107d93e6345925616f2d445397076c1c7f6b056e44aa1c99a7 2024-09-10T10:01:22.032 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:01:21 smithi045 systemd[1]: Started Ceph mds.a for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 -- 172.21.15.45:0/183903528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8103660 msgr2=0x7f28b810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 --2- 172.21.15.45:0/183903528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8103660 0x7f28b810bb50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f28a0009900 tx=0x7f28a002a480 comp rx=0 tx=0).stop 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 -- 172.21.15.45:0/183903528 shutdown_connections 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 --2- 172.21.15.45:0/183903528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b810c170 0x7f28b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 --2- 172.21.15.45:0/183903528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8103660 0x7f28b810bb50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 --2- 172.21.15.45:0/183903528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b8102d40 0x7f28b8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 -- 172.21.15.45:0/183903528 >> 172.21.15.45:0/183903528 conn(0x7f28b80fc7a0 msgr2=0x7f28b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 -- 172.21.15.45:0/183903528 shutdown_connections 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 -- 172.21.15.45:0/183903528 wait complete. 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 Processor -- start 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.368+0000 7f28beb05640 1 -- start start 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28beb05640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b8102d40 0x7f28b8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28beb05640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b8103660 0x7f28b8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28beb05640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b810c170 0x7f28b819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28beb05640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8111f70 con 0x7f28b810c170 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28beb05640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8111df0 con 0x7f28b8103660 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28beb05640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f28b81120f0 con 0x7f28b8102d40 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b810c170 0x7f28b819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28b7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b8103660 0x7f28b8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b810c170 0x7f28b819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43034/0 (socket says 172.21.15.45:43034) 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28b7fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b8103660 0x7f28b8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:34992/0 (socket says 172.21.15.45:34992) 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 -- 172.21.15.45:0/3660115568 learned_addr learned my addr 172.21.15.45:0/3660115568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bc8a1640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b8102d40 0x7f28b8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 -- 172.21.15.45:0/3660115568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b8102d40 msgr2=0x7f28b8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b8102d40 0x7f28b8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 -- 172.21.15.45:0/3660115568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b8103660 msgr2=0x7f28b8198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b8103660 0x7f28b8198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 -- 172.21.15.45:0/3660115568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b819d400 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28b7fff640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b8103660 0x7f28b8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28bd0a2640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b810c170 0x7f28b819cd20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28b000eb10 tx=0x7f28b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.369+0000 7f28b5ffb640 1 -- 172.21.15.45:0/3660115568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0004b20 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.370+0000 7f28b5ffb640 1 -- 172.21.15.45:0/3660115568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28b0004cc0 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.370+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b819d6f0 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.370+0000 7f28b5ffb640 1 -- 172.21.15.45:0/3660115568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0010760 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.370+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b81a4fd0 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.371+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b8103120 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.372+0000 7f28b5ffb640 1 -- 172.21.15.45:0/3660115568 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f28b0005020 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.373+0000 7f28b5ffb640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f28a40a2c00 0x7f28a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.373+0000 7f28bc8a1640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f28a40a2c00 0x7f28a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.373+0000 7f28b5ffb640 1 -- 172.21.15.45:0/3660115568 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f28b00cf030 con 0x7f28b810c170 2024-09-10T10:01:22.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.373+0000 7f28bc8a1640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f28a40a2c00 0x7f28a40a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f28a8005fd0 tx=0x7f28a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.377+0000 7f28b5ffb640 1 -- 172.21.15.45:0/3660115568 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28b009c430 con 0x7f28b810c170 2024-09-10T10:01:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.593+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b8106c50 con 0x7f28b810c170 2024-09-10T10:01:22.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:22 smithi045 ceph-mon[31114]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:22.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.595 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:22.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:22.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:22 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:22.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.594+0000 7f28b5ffb640 1 -- 172.21.15.45:0/3660115568 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9692 (secure 0 0 0) 0x7f28b009bd30 con 0x7f28b810c170 2024-09-10T10:01:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:22.597 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"btime":"2024-09-10T10:01:22:449065+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T10:01:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f28a40a2c00 msgr2=0x7f28a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f28a40a2c00 0x7f28a40a50c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f28a8005fd0 tx=0x7f28a800b390 comp rx=0 tx=0).stop 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b810c170 msgr2=0x7f28b819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b810c170 0x7f28b819cd20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28b000eb10 tx=0x7f28b000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 shutdown_connections 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f28a40a2c00 0x7f28a40a50c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b810c170 0x7f28b819cd20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28b8103660 0x7f28b8198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 --2- 172.21.15.45:0/3660115568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f28b8102d40 0x7f28b8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.596+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 >> 172.21.15.45:0/3660115568 conn(0x7f28b80fc7a0 msgr2=0x7f28b80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.597+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 shutdown_connections 2024-09-10T10:01:22.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:22.597+0000 7f28beb05640 1 -- 172.21.15.45:0/3660115568 wait complete. 2024-09-10T10:01:22.601 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:22 smithi047 ceph-mon[35244]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:22.601 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.601 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.601 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.601 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:22.601 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:22.601 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:22 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:22 smithi162 ceph-mon[34963]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T10:01:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T10:01:22.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:22 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:23 smithi045 ceph-mon[31114]: Deploying daemon mds.b on smithi047 2024-09-10T10:01:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:23 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.45:6840/511130082,v1:172.21.15.45:6841/511130082] up:boot 2024-09-10T10:01:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:23 smithi045 ceph-mon[31114]: fsmap 11 up:standby 2024-09-10T10:01:23.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:23 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:01:23.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:23 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3660115568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:23 smithi047 ceph-mon[35244]: Deploying daemon mds.b on smithi047 2024-09-10T10:01:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:23 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.45:6840/511130082,v1:172.21.15.45:6841/511130082] up:boot 2024-09-10T10:01:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:23 smithi047 ceph-mon[35244]: fsmap 11 up:standby 2024-09-10T10:01:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:23 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:01:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:23 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3660115568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:23 smithi162 ceph-mon[34963]: Deploying daemon mds.b on smithi047 2024-09-10T10:01:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:23 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.45:6840/511130082,v1:172.21.15.45:6841/511130082] up:boot 2024-09-10T10:01:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:23 smithi162 ceph-mon[34963]: fsmap 11 up:standby 2024-09-10T10:01:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:23 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:01:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:23 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3660115568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:24.058 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:23 smithi047 systemd[1]: Starting Ceph mds.b for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:01:24.323 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 podman[66126]: 2024-09-10 10:01:23.960057021 +0000 UTC m=+0.016141036 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:01:24.323 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 podman[66126]: 2024-09-10 10:01:24.079798222 +0000 UTC m=+0.135882232 container create f6d7311f537f63baa6e95a8e2ab1b23ec29ceeac20f123c97ee90081b4228c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T10:01:24.323 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 podman[66126]: 2024-09-10 10:01:24.322573917 +0000 UTC m=+0.378657933 container init f6d7311f537f63baa6e95a8e2ab1b23ec29ceeac20f123c97ee90081b4228c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T10:01:24.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:24 smithi047 ceph-mon[35244]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:24.660 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 podman[66126]: 2024-09-10 10:01:24.325549452 +0000 UTC m=+0.381633464 container start f6d7311f537f63baa6e95a8e2ab1b23ec29ceeac20f123c97ee90081b4228c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T10:01:24.660 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 ceph-mds[66145]: -- 172.21.15.47:0/1351788919 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f8bbfb860 con 0x559f8bbd9400 2024-09-10T10:01:24.660 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b[66141]: starting mds.b at 2024-09-10T10:01:24.660 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 bash[66126]: f6d7311f537f63baa6e95a8e2ab1b23ec29ceeac20f123c97ee90081b4228c30 2024-09-10T10:01:24.660 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:24 smithi047 systemd[1]: Started Ceph mds.b for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:01:24.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:24 smithi162 ceph-mon[34963]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:24.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:24 smithi045 ceph-mon[31114]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:25.353 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] up:boot 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: fsmap 12 up:standby 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:25 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:25.842 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] up:boot 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: fsmap 12 up:standby 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:25 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] up:boot 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: fsmap 12 up:standby 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:25.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:25 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 -- 172.21.15.45:0/3004846285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c102790 msgr2=0x7f973c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 --2- 172.21.15.45:0/3004846285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c102790 0x7f973c1067e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f972c009900 tx=0x7f972c02a410 comp rx=0 tx=0).stop 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 -- 172.21.15.45:0/3004846285 shutdown_connections 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 --2- 172.21.15.45:0/3004846285 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f973c106f10 0x7f973c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 --2- 172.21.15.45:0/3004846285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c102790 0x7f973c1067e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 --2- 172.21.15.45:0/3004846285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f973c101de0 0x7f973c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 -- 172.21.15.45:0/3004846285 >> 172.21.15.45:0/3004846285 conn(0x7f973c0fd8c0 msgr2=0x7f973c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 -- 172.21.15.45:0/3004846285 shutdown_connections 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 -- 172.21.15.45:0/3004846285 wait complete. 2024-09-10T10:01:26.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 Processor -- start 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.179+0000 7f9743553640 1 -- start start 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9743553640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f973c101de0 0x7f973c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9743553640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f973c102790 0x7f973c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9743553640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c106f10 0x7f973c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9743553640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f973c111f70 con 0x7f973c106f10 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9743553640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f973c111df0 con 0x7f973c102790 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9743553640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f973c1120f0 con 0x7f973c101de0 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9741af0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c106f10 0x7f973c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97412ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f973c101de0 0x7f973c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9741af0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c106f10 0x7f973c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39474/0 (socket says 172.21.15.45:39474) 2024-09-10T10:01:26.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9741af0640 1 -- 172.21.15.45:0/3796580198 learned_addr learned my addr 172.21.15.45:0/3796580198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9740aee640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f973c102790 0x7f973c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97412ef640 1 -- 172.21.15.45:0/3796580198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f973c102790 msgr2=0x7f973c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97412ef640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f973c102790 0x7f973c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97412ef640 1 -- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c106f10 msgr2=0x7f973c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97412ef640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c106f10 0x7f973c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97412ef640 1 -- 172.21.15.45:0/3796580198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973c19d3c0 con 0x7f973c101de0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9741af0640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c106f10 0x7f973c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97412ef640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f973c101de0 0x7f973c198410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f973800e9f0 tx=0x7f973800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97327fc640 1 -- 172.21.15.45:0/3796580198 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800ccc0 con 0x7f973c101de0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973c19d6b0 con 0x7f973c101de0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.180+0000 7f97327fc640 1 -- 172.21.15.45:0/3796580198 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9738010910 con 0x7f973c101de0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.181+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f973c1a4f90 con 0x7f973c101de0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.181+0000 7f97327fc640 1 -- 172.21.15.45:0/3796580198 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738018900 con 0x7f973c101de0 2024-09-10T10:01:26.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.181+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973c1021c0 con 0x7f973c101de0 2024-09-10T10:01:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.183+0000 7f97327fc640 1 -- 172.21.15.45:0/3796580198 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9738010ac0 con 0x7f973c101de0 2024-09-10T10:01:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.183+0000 7f97327fc640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f97280a2b80 0x7f97280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.183+0000 7f9740aee640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f97280a2b80 0x7f97280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.183+0000 7f97327fc640 1 -- 172.21.15.45:0/3796580198 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f97380cf2e0 con 0x7f973c101de0 2024-09-10T10:01:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.184+0000 7f9740aee640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f97280a2b80 0x7f97280a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f972c009230 tx=0x7f972c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.186+0000 7f97327fc640 1 -- 172.21.15.45:0/3796580198 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f973809bff0 con 0x7f973c101de0 2024-09-10T10:01:26.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.423+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f973c10a180 con 0x7f973c101de0 2024-09-10T10:01:26.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.424+0000 7f97327fc640 1 -- 172.21.15.45:0/3796580198 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7f973807abc0 con 0x7f973c101de0 2024-09-10T10:01:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:26.426 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"btime":"2024-09-10T10:01:24:461306+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T10:01:26.426 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2024-09-10T10:01:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f97280a2b80 msgr2=0x7f97280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f97280a2b80 0x7f97280a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f972c009230 tx=0x7f972c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f973c101de0 msgr2=0x7f973c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f973c101de0 0x7f973c198410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f973800e9f0 tx=0x7f973800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 shutdown_connections 2024-09-10T10:01:26.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f97280a2b80 0x7f97280a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f973c106f10 0x7f973c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f973c102790 0x7f973c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 --2- 172.21.15.45:0/3796580198 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f973c101de0 0x7f973c198410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 >> 172.21.15.45:0/3796580198 conn(0x7f973c0fd8c0 msgr2=0x7f973c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 shutdown_connections 2024-09-10T10:01:26.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:26.427+0000 7f9743553640 1 -- 172.21.15.45:0/3796580198 wait complete. 2024-09-10T10:01:26.708 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:26 smithi045 ceph-mon[31114]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:26.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:26 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:26 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:26 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:26 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.709 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:26 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3796580198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:26.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:26 smithi162 ceph-mon[34963]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:26.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:26 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:26 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:26 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:26 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:26 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3796580198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:26.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:26 smithi047 ceph-mon[35244]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:26.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:26 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:26 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:26 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:26 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:26.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:26 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3796580198' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:27.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T10:01:27.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 -- 172.21.15.45:0/700685029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c103760 msgr2=0x7f880c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 --2- 172.21.15.45:0/700685029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c103760 0x7f880c107ba0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009930 tx=0x7f87f802a420 comp rx=0 tx=0).stop 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 -- 172.21.15.45:0/700685029 shutdown_connections 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 --2- 172.21.15.45:0/700685029 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f880c1082e0 0x7f880c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 --2- 172.21.15.45:0/700685029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c103760 0x7f880c107ba0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 --2- 172.21.15.45:0/700685029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c102db0 0x7f880c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 -- 172.21.15.45:0/700685029 >> 172.21.15.45:0/700685029 conn(0x7f880c0fc7d0 msgr2=0x7f880c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 -- 172.21.15.45:0/700685029 shutdown_connections 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.896+0000 7f8812c19640 1 -- 172.21.15.45:0/700685029 wait complete. 2024-09-10T10:01:27.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 Processor -- start 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 -- start start 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c102db0 0x7f880c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f880c103760 0x7f880c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c1082e0 0x7f880c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f880c112070 con 0x7f880c1082e0 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f880c111ef0 con 0x7f880c102db0 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f8812c19640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1121f0 con 0x7f880c103760 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f88111b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c1082e0 0x7f880c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f88111b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c1082e0 0x7f880c19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39502/0 (socket says 172.21.15.45:39502) 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f88111b6640 1 -- 172.21.15.45:0/1253208229 learned_addr learned my addr 172.21.15.45:0/1253208229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.897+0000 7f880bfff640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f880c103760 0x7f880c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88109b5640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c102db0 0x7f880c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88111b6640 1 -- 172.21.15.45:0/1253208229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f880c103760 msgr2=0x7f880c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88111b6640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f880c103760 0x7f880c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88111b6640 1 -- 172.21.15.45:0/1253208229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c102db0 msgr2=0x7f880c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88111b6640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c102db0 0x7f880c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88111b6640 1 -- 172.21.15.45:0/1253208229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c19d580 con 0x7f880c1082e0 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88109b5640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c102db0 0x7f880c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f880bfff640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f880c103760 0x7f880c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f88111b6640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c1082e0 0x7f880c19cea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f880400eb10 tx=0x7f880400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f8809ffb640 1 -- 172.21.15.45:0/1253208229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804004b20 con 0x7f880c1082e0 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f8809ffb640 1 -- 172.21.15.45:0/1253208229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804004cc0 con 0x7f880c1082e0 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c19d870 con 0x7f880c1082e0 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.898+0000 7f8809ffb640 1 -- 172.21.15.45:0/1253208229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804010760 con 0x7f880c1082e0 2024-09-10T10:01:27.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.899+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1a5150 con 0x7f880c1082e0 2024-09-10T10:01:27.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.900+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c103190 con 0x7f880c1082e0 2024-09-10T10:01:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.901+0000 7f8809ffb640 1 -- 172.21.15.45:0/1253208229 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8804005020 con 0x7f880c1082e0 2024-09-10T10:01:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.901+0000 7f8809ffb640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87fc0a2c00 0x7f87fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.902+0000 7f88109b5640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87fc0a2c00 0x7f87fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.902+0000 7f8809ffb640 1 -- 172.21.15.45:0/1253208229 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f88040cf150 con 0x7f880c1082e0 2024-09-10T10:01:27.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.902+0000 7f88109b5640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87fc0a2c00 0x7f87fc0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fc0 tx=0x7f8800005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:27.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:27.906+0000 7f8809ffb640 1 -- 172.21.15.45:0/1253208229 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f880409c550 con 0x7f880c1082e0 2024-09-10T10:01:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.099+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f880c10a2c0 con 0x7f880c1082e0 2024-09-10T10:01:28.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.100+0000 7f8809ffb640 1 -- 172.21.15.45:0/1253208229 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f880409be50 con 0x7f880c1082e0 2024-09-10T10:01:28.101 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:28.101 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-09-10T10:01:28.101 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:01:28.101 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-09-10T10:01:28.101 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-09-10T10:01:28.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:01:28.102 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87fc0a2c00 msgr2=0x7f87fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87fc0a2c00 0x7f87fc0a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fc0 tx=0x7f8800005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c1082e0 msgr2=0x7f880c19cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c1082e0 0x7f880c19cea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f880400eb10 tx=0x7f880400c2d0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 shutdown_connections 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f87fc0a2c00 0x7f87fc0a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f880c1082e0 0x7f880c19cea0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f880c103760 0x7f880c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 --2- 172.21.15.45:0/1253208229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c102db0 0x7f880c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 >> 172.21.15.45:0/1253208229 conn(0x7f880c0fc7d0 msgr2=0x7f880c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 shutdown_connections 2024-09-10T10:01:28.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:28.102+0000 7f8812c19640 1 -- 172.21.15.45:0/1253208229 wait complete. 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:28.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:28 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1253208229' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:01:28.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:28.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:28.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:28.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:28.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:28 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1253208229' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:01:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:28 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1253208229' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T10:01:29.378 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:01:29.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:01:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:29 smithi045 ceph-mon[31114]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:29 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.651 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:29.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:29 smithi162 ceph-mon[34963]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:29.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:29 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:29 smithi047 ceph-mon[35244]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:29.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:29.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:29 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 -- 172.21.15.45:0/2903333589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 msgr2=0x7f5a40100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 --2- 172.21.15.45:0/2903333589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40100ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009930 tx=0x7f5a3402a420 comp rx=0 tx=0).stop 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 -- 172.21.15.45:0/2903333589 shutdown_connections 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 --2- 172.21.15.45:0/2903333589 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a40101420 0x7f5a4010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 --2- 172.21.15.45:0/2903333589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40100ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 --2- 172.21.15.45:0/2903333589 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a40106a80 0x7f5a40106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 -- 172.21.15.45:0/2903333589 >> 172.21.15.45:0/2903333589 conn(0x7f5a400fc7a0 msgr2=0x7f5a400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 -- 172.21.15.45:0/2903333589 shutdown_connections 2024-09-10T10:01:30.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.085+0000 7f5a45168640 1 -- 172.21.15.45:0/2903333589 wait complete. 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 Processor -- start 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 -- start start 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a40101420 0x7f5a40198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a40106a80 0x7f5a4019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40110e80 con 0x7f5a40100a80 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40110d00 con 0x7f5a40101420 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a45168640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40111000 con 0x7f5a40106a80 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a3ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.086+0000 7f5a3e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a40101420 0x7f5a40198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39530/0 (socket says 172.21.15.45:39530) 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 -- 172.21.15.45:0/3483062460 learned_addr learned my addr 172.21.15.45:0/3483062460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:30.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3f59e640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a40106a80 0x7f5a4019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 -- 172.21.15.45:0/3483062460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a40106a80 msgr2=0x7f5a4019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a40106a80 0x7f5a4019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 -- 172.21.15.45:0/3483062460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a40101420 msgr2=0x7f5a40198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a40101420 0x7f5a40198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 -- 172.21.15.45:0/3483062460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4019d3e0 con 0x7f5a40100a80 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3e59c640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a40101420 0x7f5a40198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a3ed9d640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40198430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9f0 tx=0x7f5a3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a27fff640 1 -- 172.21.15.45:0/3483062460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000cd80 con 0x7f5a40100a80 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4019d6d0 con 0x7f5a40100a80 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a27fff640 1 -- 172.21.15.45:0/3483062460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a30010910 con 0x7f5a40100a80 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a4fb0 con 0x7f5a40100a80 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.087+0000 7f5a27fff640 1 -- 172.21.15.45:0/3483062460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30018910 con 0x7f5a40100a80 2024-09-10T10:01:30.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.088+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a40100ee0 con 0x7f5a40100a80 2024-09-10T10:01:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.089+0000 7f5a27fff640 1 -- 172.21.15.45:0/3483062460 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a30010ac0 con 0x7f5a40100a80 2024-09-10T10:01:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.090+0000 7f5a27fff640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5a280a2b80 0x7f5a280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.090+0000 7f5a3e59c640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5a280a2b80 0x7f5a280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.090+0000 7f5a27fff640 1 -- 172.21.15.45:0/3483062460 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5a300ceef0 con 0x7f5a40100a80 2024-09-10T10:01:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.091+0000 7f5a3e59c640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5a280a2b80 0x7f5a280a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009900 tx=0x7f5a340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.094+0000 7f5a27fff640 1 -- 172.21.15.45:0/3483062460 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a3009bc00 con 0x7f5a40100a80 2024-09-10T10:01:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:30.275+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5a4010a020 con 0x7f5a40100a80 2024-09-10T10:01:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.187+0000 7f5a27fff640 1 -- 172.21.15.45:0/3483062460 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f5a3009ff00 con 0x7f5a40100a80 2024-09-10T10:01:31.189 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5a280a2b80 msgr2=0x7f5a280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5a280a2b80 0x7f5a280a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009900 tx=0x7f5a340058d0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 msgr2=0x7f5a40198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40198430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9f0 tx=0x7f5a3000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 shutdown_connections 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5a280a2b80 0x7f5a280a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5a40106a80 0x7f5a4019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a40101420 0x7f5a40198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 --2- 172.21.15.45:0/3483062460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a40100a80 0x7f5a40198430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.190+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 >> 172.21.15.45:0/3483062460 conn(0x7f5a400fc7a0 msgr2=0x7f5a4006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.191+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 shutdown_connections 2024-09-10T10:01:31.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:31.191+0000 7f5a45168640 1 -- 172.21.15.45:0/3483062460 wait complete. 2024-09-10T10:01:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:31 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:31 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3483062460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:01:31.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:31.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:31.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:31 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.572 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:31 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3483062460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:01:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:31 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:01:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:31 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3483062460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T10:01:31.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:01:32.130 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:32.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:32 smithi045 ceph-mon[31114]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:32.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:32 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3483062460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:01:32.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:32 smithi045 ceph-mon[31114]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 -- 172.21.15.45:0/356561327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620102ef0 msgr2=0x7f362010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 --2- 172.21.15.45:0/356561327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620102ef0 0x7f362010da40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3618009ca0 tx=0x7f361802a950 comp rx=0 tx=0).stop 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 -- 172.21.15.45:0/356561327 shutdown_connections 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 --2- 172.21.15.45:0/356561327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620102ef0 0x7f362010da40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 --2- 172.21.15.45:0/356561327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3620102570 0x7f36201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 --2- 172.21.15.45:0/356561327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3620100e20 0x7f3620101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 -- 172.21.15.45:0/356561327 >> 172.21.15.45:0/356561327 conn(0x7f36200fc7d0 msgr2=0x7f36200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:32.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 -- 172.21.15.45:0/356561327 shutdown_connections 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 -- 172.21.15.45:0/356561327 wait complete. 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 Processor -- start 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.602+0000 7f3625f6a640 1 -- start start 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620100e20 0x7f36201984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3620102570 0x7f3620198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3620102ef0 0x7f362019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36201120a0 con 0x7f3620100e20 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3620111f20 con 0x7f3620102ef0 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3620112220 con 0x7f3620102570 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620100e20 0x7f36201984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620100e20 0x7f36201984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39558/0 (socket says 172.21.15.45:39558) 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 -- 172.21.15.45:0/1796352191 learned_addr learned my addr 172.21.15.45:0/1796352191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:32.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 -- 172.21.15.45:0/1796352191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3620102570 msgr2=0x7f3620198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3620102570 0x7f3620198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 -- 172.21.15.45:0/1796352191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3620102ef0 msgr2=0x7f362019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361ffff640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3620102ef0 0x7f362019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3620102ef0 0x7f362019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 -- 172.21.15.45:0/1796352191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362019d470 con 0x7f3620100e20 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361f7fe640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620100e20 0x7f36201984c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f361000d950 tx=0x7f361000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361cff9640 1 -- 172.21.15.45:0/1796352191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610017030 con 0x7f3620100e20 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362019d760 con 0x7f3620100e20 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361cff9640 1 -- 172.21.15.45:0/1796352191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3610004020 con 0x7f3620100e20 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f361cff9640 1 -- 172.21.15.45:0/1796352191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36100047a0 con 0x7f3620100e20 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.603+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36201a5040 con 0x7f3620100e20 2024-09-10T10:01:32.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.604+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3620101200 con 0x7f3620100e20 2024-09-10T10:01:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.606+0000 7f361cff9640 1 -- 172.21.15.45:0/1796352191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36100041d0 con 0x7f3620100e20 2024-09-10T10:01:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.606+0000 7f361cff9640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36080a2b80 0x7f36080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.606+0000 7f361effd640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36080a2b80 0x7f36080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.607+0000 7f361cff9640 1 -- 172.21.15.45:0/1796352191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f36100ceec0 con 0x7f3620100e20 2024-09-10T10:01:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.607+0000 7f361effd640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36080a2b80 0x7f36080a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3614005d80 tx=0x7f3614005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:32.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.610+0000 7f361cff9640 1 -- 172.21.15.45:0/1796352191 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f361009c150 con 0x7f3620100e20 2024-09-10T10:01:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:32 smithi162 ceph-mon[34963]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:32 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3483062460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:01:32.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:32 smithi162 ceph-mon[34963]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:01:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:32 smithi047 ceph-mon[35244]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:32 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3483062460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T10:01:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:32 smithi047 ceph-mon[35244]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T10:01:32.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:32.786+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3620199820 con 0x7f3620100e20 2024-09-10T10:01:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.224+0000 7f361cff9640 1 -- 172.21.15.45:0/1796352191 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f361009ba50 con 0x7f3620100e20 2024-09-10T10:01:33.225 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-09-10T10:01:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36080a2b80 msgr2=0x7f36080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36080a2b80 0x7f36080a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3614005d80 tx=0x7f3614005c90 comp rx=0 tx=0).stop 2024-09-10T10:01:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620100e20 msgr2=0x7f36201984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620100e20 0x7f36201984c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f361000d950 tx=0x7f361000de10 comp rx=0 tx=0).stop 2024-09-10T10:01:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 shutdown_connections 2024-09-10T10:01:33.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36080a2b80 0x7f36080a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3620102ef0 0x7f362019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3620102570 0x7f3620198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 --2- 172.21.15.45:0/1796352191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3620100e20 0x7f36201984c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 >> 172.21.15.45:0/1796352191 conn(0x7f36200fc7d0 msgr2=0x7f36200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 shutdown_connections 2024-09-10T10:01:33.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:33.227+0000 7f3625f6a640 1 -- 172.21.15.45:0/1796352191 wait complete. 2024-09-10T10:01:33.606 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:33 smithi045 ceph-mon[31114]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:01:33.606 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:33 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1796352191' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:01:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:33 smithi162 ceph-mon[34963]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:01:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:33 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1796352191' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:01:33.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:33 smithi047 ceph-mon[35244]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T10:01:33.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:33 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1796352191' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T10:01:33.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:01:34.221 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:34 smithi045 ceph-mon[31114]: pgmap v270: 33 pgs: 2 creating+activating, 3 creating+peering, 21 unknown, 7 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:34.496 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:34 smithi045 ceph-mon[31114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:01:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:34 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1796352191' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:01:34.497 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:34 smithi045 ceph-mon[31114]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:01:34.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:34 smithi162 ceph-mon[34963]: pgmap v270: 33 pgs: 2 creating+activating, 3 creating+peering, 21 unknown, 7 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:34 smithi162 ceph-mon[34963]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:01:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:34 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1796352191' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:01:34.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:34 smithi162 ceph-mon[34963]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:01:34.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:34 smithi047 ceph-mon[35244]: pgmap v270: 33 pgs: 2 creating+activating, 3 creating+peering, 21 unknown, 7 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:34.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:34 smithi047 ceph-mon[35244]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T10:01:34.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:34 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1796352191' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T10:01:34.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:34 smithi047 ceph-mon[35244]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T10:01:34.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 -- 172.21.15.45:0/3022005763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c581037a0 msgr2=0x7f5c5810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/3022005763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c581037a0 0x7f5c5810bc00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009ca0 tx=0x7f5c4802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 -- 172.21.15.45:0/3022005763 shutdown_connections 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/3022005763 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c5810c2f0 0x7f5c5810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/3022005763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c581037a0 0x7f5c5810bc00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/3022005763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c58102df0 0x7f5c581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 -- 172.21.15.45:0/3022005763 >> 172.21.15.45:0/3022005763 conn(0x7f5c580fc7d0 msgr2=0x7f5c580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 -- 172.21.15.45:0/3022005763 shutdown_connections 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.702+0000 7f5c5ead0640 1 -- 172.21.15.45:0/3022005763 wait complete. 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 Processor -- start 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 -- start start 2024-09-10T10:01:34.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c58102df0 0x7f5c58198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c581037a0 0x7f5c58198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c5810c2f0 0x7f5c5819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58112130 con 0x7f5c58102df0 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5c58111fb0 con 0x7f5c5810c2f0 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5ead0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5c581122b0 con 0x7f5c581037a0 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5c86c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c58102df0 0x7f5c58198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5c86c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c58102df0 0x7f5c58198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41238/0 (socket says 172.21.15.45:41238) 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c5c86c640 1 -- 172.21.15.45:0/2013534291 learned_addr learned my addr 172.21.15.45:0/2013534291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.703+0000 7f5c57fff640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c581037a0 0x7f5c58198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5d06d640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c5810c2f0 0x7f5c5819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5c86c640 1 -- 172.21.15.45:0/2013534291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c581037a0 msgr2=0x7f5c58198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5c86c640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c581037a0 0x7f5c58198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5c86c640 1 -- 172.21.15.45:0/2013534291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c5810c2f0 msgr2=0x7f5c5819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5c86c640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c5810c2f0 0x7f5c5819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5c86c640 1 -- 172.21.15.45:0/2013534291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5819c6c0 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5d06d640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c5810c2f0 0x7f5c5819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5c86c640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c58102df0 0x7f5c58198770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c890 tx=0x7f5c4000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c55ffb640 1 -- 172.21.15.45:0/2013534291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c40004220 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c55ffb640 1 -- 172.21.15.45:0/2013534291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c400043c0 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c55ffb640 1 -- 172.21.15.45:0/2013534291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c400049e0 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5819c9b0 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.704+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c581a5320 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.705+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c581031d0 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.706+0000 7f5c55ffb640 1 -- 172.21.15.45:0/2013534291 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5c400079b0 con 0x7f5c58102df0 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.707+0000 7f5c55ffb640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5c440a2b80 0x7f5c440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.707+0000 7f5c57fff640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5c440a2b80 0x7f5c440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.707+0000 7f5c55ffb640 1 -- 172.21.15.45:0/2013534291 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f5c400cefa0 con 0x7f5c58102df0 2024-09-10T10:01:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.707+0000 7f5c57fff640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5c440a2b80 0x7f5c440a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c4802f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.712+0000 7f5c55ffb640 1 -- 172.21.15.45:0/2013534291 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c40014030 con 0x7f5c58102df0 2024-09-10T10:01:34.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:34.923+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5c58199ad0 con 0x7f5c58102df0 2024-09-10T10:01:35.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.236+0000 7f5c55ffb640 1 -- 172.21.15.45:0/2013534291 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5c4009c0b0 con 0x7f5c58102df0 2024-09-10T10:01:35.237 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T10:01:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.238+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5c440a2b80 msgr2=0x7f5c440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.238+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5c440a2b80 0x7f5c440a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c4802f310 comp rx=0 tx=0).stop 2024-09-10T10:01:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.238+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c58102df0 msgr2=0x7f5c58198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.238+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c58102df0 0x7f5c58198770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000c890 tx=0x7f5c4000cd50 comp rx=0 tx=0).stop 2024-09-10T10:01:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 shutdown_connections 2024-09-10T10:01:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f5c440a2b80 0x7f5c440a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:35.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c5810c2f0 0x7f5c5819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c581037a0 0x7f5c58198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 --2- 172.21.15.45:0/2013534291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c58102df0 0x7f5c58198770 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 >> 172.21.15.45:0/2013534291 conn(0x7f5c580fc7d0 msgr2=0x7f5c580fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 shutdown_connections 2024-09-10T10:01:35.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:35.239+0000 7f5c5ead0640 1 -- 172.21.15.45:0/2013534291 wait complete. 2024-09-10T10:01:35.613 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:35 smithi045 ceph-mon[31114]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:01:35.614 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:35 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2013534291' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:01:35.614 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:35 smithi045 ceph-mon[31114]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T10:01:35.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:35 smithi162 ceph-mon[34963]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:01:35.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:35 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2013534291' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:01:35.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:35 smithi162 ceph-mon[34963]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T10:01:35.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:35 smithi047 ceph-mon[35244]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T10:01:35.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:35 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2013534291' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T10:01:35.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:35 smithi047 ceph-mon[35244]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T10:01:35.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:01:36.187 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:36 smithi045 ceph-mon[31114]: pgmap v273: 97 pgs: 2 creating+activating, 39 creating+peering, 28 unknown, 28 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:36 smithi045 ceph-mon[31114]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:01:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:36 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2013534291' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:01:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:36 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T10:01:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:36 smithi045 ceph-mon[31114]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T10:01:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:36 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] up:active 2024-09-10T10:01:36.551 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:36 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.630+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/206725484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4100a80 msgr2=0x7fe7c4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.630+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/206725484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4100a80 0x7fe7c4100ee0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009930 tx=0x7fe7b802a420 comp rx=0 tx=0).stop 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.630+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/206725484 shutdown_connections 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.630+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/206725484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c4101420 0x7fe7c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.630+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/206725484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4100a80 0x7fe7c4100ee0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.630+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/206725484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7c4106a80 0x7fe7c4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.630+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/206725484 >> 172.21.15.45:0/206725484 conn(0x7fe7c40fc7a0 msgr2=0x7fe7c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/206725484 shutdown_connections 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/206725484 wait complete. 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 Processor -- start 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 -- start start 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c4100a80 0x7fe7c4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4101420 0x7fe7c41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7c4106a80 0x7fe7c419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c4111fa0 con 0x7fe7c4101420 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c4111e20 con 0x7fe7c4100a80 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.631+0000 7fe7c9c4b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c4112120 con 0x7fe7c4106a80 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4101420 0x7fe7c41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4101420 0x7fe7c41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41268/0 (socket says 172.21.15.45:41268) 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 -- 172.21.15.45:0/2980043659 learned_addr learned my addr 172.21.15.45:0/2980043659 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c3fff640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7c4106a80 0x7fe7c419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c37fe640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c4100a80 0x7fe7c4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 -- 172.21.15.45:0/2980043659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7c4106a80 msgr2=0x7fe7c419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7c4106a80 0x7fe7c419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 -- 172.21.15.45:0/2980043659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c4100a80 msgr2=0x7fe7c4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c4100a80 0x7fe7c4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 -- 172.21.15.45:0/2980043659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c419d430 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c2ffd640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4101420 0x7fe7c41989c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009230 tx=0x7fe7b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c0ff9640 1 -- 172.21.15.45:0/2980043659 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b803f030 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c0ff9640 1 -- 172.21.15.45:0/2980043659 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b8003ad0 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c419d6c0 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c0ff9640 1 -- 172.21.15.45:0/2980043659 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8035070 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.632+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c41a5000 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.634+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c4100ee0 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.635+0000 7fe7c0ff9640 1 -- 172.21.15.45:0/2980043659 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe7b8042030 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.635+0000 7fe7c0ff9640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe7ac0a2c00 0x7fe7ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.635+0000 7fe7c37fe640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe7ac0a2c00 0x7fe7ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.635+0000 7fe7c0ff9640 1 -- 172.21.15.45:0/2980043659 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fe7b80ef2d0 con 0x7fe7c4101420 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.636+0000 7fe7c37fe640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe7ac0a2c00 0x7fe7ac0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400bb60 tx=0x7fe7b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:36.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.640+0000 7fe7c0ff9640 1 -- 172.21.15.45:0/2980043659 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7b80bbd30 con 0x7fe7c4101420 2024-09-10T10:01:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:36 smithi162 ceph-mon[34963]: pgmap v273: 97 pgs: 2 creating+activating, 39 creating+peering, 28 unknown, 28 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:36 smithi162 ceph-mon[34963]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:01:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:36 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2013534291' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:01:36.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:36 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T10:01:36.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:36 smithi162 ceph-mon[34963]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T10:01:36.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:36 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] up:active 2024-09-10T10:01:36.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:36 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:36 smithi047 ceph-mon[35244]: pgmap v273: 97 pgs: 2 creating+activating, 39 creating+peering, 28 unknown, 28 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:36 smithi047 ceph-mon[35244]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T10:01:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:36 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2013534291' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T10:01:36.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:36 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T10:01:36.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:36 smithi047 ceph-mon[35244]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T10:01:36.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:36 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] up:active 2024-09-10T10:01:36.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:36 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:36.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.815+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7c4109020 con 0x7fe7c4101420 2024-09-10T10:01:36.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.817+0000 7fe7c0ff9640 1 -- 172.21.15.45:0/2980043659 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22242 (secure 0 0 0) 0x7fe7b80c0030 con 0x7fe7c4101420 2024-09-10T10:01:36.818 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:36.819 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:01:34.924685+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:36.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe7ac0a2c00 msgr2=0x7fe7ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe7ac0a2c00 0x7fe7ac0a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400bb60 tx=0x7fe7b400b390 comp rx=0 tx=0).stop 2024-09-10T10:01:36.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4101420 msgr2=0x7fe7c41989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:36.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4101420 0x7fe7c41989c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009230 tx=0x7fe7b8004270 comp rx=0 tx=0).stop 2024-09-10T10:01:36.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 shutdown_connections 2024-09-10T10:01:36.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe7ac0a2c00 0x7fe7ac0a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe7c4106a80 0x7fe7c419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7c4101420 0x7fe7c41989c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 --2- 172.21.15.45:0/2980043659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7c4100a80 0x7fe7c4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:36.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 >> 172.21.15.45:0/2980043659 conn(0x7fe7c40fc7a0 msgr2=0x7fe7c40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:36.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 shutdown_connections 2024-09-10T10:01:36.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:36.819+0000 7fe7c9c4b640 1 -- 172.21.15.45:0/2980043659 wait complete. 2024-09-10T10:01:37.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:01:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:37 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2980043659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:37 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2980043659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:37 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2980043659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:37.718 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 -- 172.21.15.45:0/990885388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 msgr2=0x7f0c381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 --2- 172.21.15.45:0/990885388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c381067e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009900 tx=0x7f0c3402a410 comp rx=0 tx=0).stop 2024-09-10T10:01:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 -- 172.21.15.45:0/990885388 shutdown_connections 2024-09-10T10:01:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 --2- 172.21.15.45:0/990885388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38106f10 0x7f0c3810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 --2- 172.21.15.45:0/990885388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c381067e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 --2- 172.21.15.45:0/990885388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c38101de0 0x7f0c381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 -- 172.21.15.45:0/990885388 >> 172.21.15.45:0/990885388 conn(0x7f0c380fd8c0 msgr2=0x7f0c380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 -- 172.21.15.45:0/990885388 shutdown_connections 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.164+0000 7f0c3f842640 1 -- 172.21.15.45:0/990885388 wait complete. 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 Processor -- start 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 -- start start 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38101de0 0x7f0c380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c38106f10 0x7f0c38074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112010 con 0x7f0c38102790 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38111e90 con 0x7f0c38101de0 2024-09-10T10:01:38.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3f842640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38112190 con 0x7f0c38106f10 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.165+0000 7f0c3cddd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c38074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41286/0 (socket says 172.21.15.45:41286) 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 -- 172.21.15.45:0/1269752429 learned_addr learned my addr 172.21.15.45:0/1269752429 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3dddf640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c38106f10 0x7f0c38074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3d5de640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38101de0 0x7f0c380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 -- 172.21.15.45:0/1269752429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c38106f10 msgr2=0x7f0c38074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c38106f10 0x7f0c38074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 -- 172.21.15.45:0/1269752429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38101de0 msgr2=0x7f0c380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38101de0 0x7f0c380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 -- 172.21.15.45:0/1269752429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c381a6a70 con 0x7f0c38102790 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3cddd640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c38074a30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005ab0 tx=0x7f0c340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1269752429 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3403f030 con 0x7f0c38102790 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1269752429 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c34003ce0 con 0x7f0c38102790 2024-09-10T10:01:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c381a6d00 con 0x7f0c38102790 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1269752429 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c34035070 con 0x7f0c38102790 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.166+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c381a7190 con 0x7f0c38102790 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.167+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c38102240 con 0x7f0c38102790 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.169+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1269752429 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0c34042030 con 0x7f0c38102790 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.169+0000 7f0c2a7fc640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0c240a2b80 0x7f0c240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.169+0000 7f0c3d5de640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0c240a2b80 0x7f0c240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.169+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1269752429 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f0c340ef990 con 0x7f0c38102790 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.170+0000 7f0c3d5de640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0c240a2b80 0x7f0c240a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009990 tx=0x7f0c2c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.172+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1269752429 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c340bca40 con 0x7f0c38102790 2024-09-10T10:01:38.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.350+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c3802d7c0 con 0x7f0c38102790 2024-09-10T10:01:38.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.352+0000 7f0c2a7fc640 1 -- 172.21.15.45:0/1269752429 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22242 (secure 0 0 0) 0x7f0c340a1700 con 0x7f0c38102790 2024-09-10T10:01:38.353 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:38.353 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":81,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:01:34.924685+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0c240a2b80 msgr2=0x7f0c240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0c240a2b80 0x7f0c240a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009990 tx=0x7f0c2c009110 comp rx=0 tx=0).stop 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 msgr2=0x7f0c38074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c38074a30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c34005ab0 tx=0x7f0c340042f0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 shutdown_connections 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0c240a2b80 0x7f0c240a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0c38106f10 0x7f0c38074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c38102790 0x7f0c38074a30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 --2- 172.21.15.45:0/1269752429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c38101de0 0x7f0c380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 >> 172.21.15.45:0/1269752429 conn(0x7f0c380fd8c0 msgr2=0x7f0c380fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 shutdown_connections 2024-09-10T10:01:38.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:38.354+0000 7f0c3f842640 1 -- 172.21.15.45:0/1269752429 wait complete. 2024-09-10T10:01:38.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:38 smithi162 ceph-mon[34963]: pgmap v275: 97 pgs: 2 creating+activating, 39 creating+peering, 28 unknown, 28 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:38.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:38 smithi162 ceph-mon[34963]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:01:38.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:38 smithi162 ceph-mon[34963]: Cluster is now healthy 2024-09-10T10:01:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:38 smithi045 ceph-mon[31114]: pgmap v275: 97 pgs: 2 creating+activating, 39 creating+peering, 28 unknown, 28 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:38 smithi045 ceph-mon[31114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:01:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:38 smithi045 ceph-mon[31114]: Cluster is now healthy 2024-09-10T10:01:38.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:38 smithi047 ceph-mon[35244]: pgmap v275: 97 pgs: 2 creating+activating, 39 creating+peering, 28 unknown, 28 active+clean; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:01:38.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:38 smithi047 ceph-mon[35244]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T10:01:38.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:38 smithi047 ceph-mon[35244]: Cluster is now healthy 2024-09-10T10:01:39.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:01:39.289 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:39.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:39 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1269752429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:39.608 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:39 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1269752429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:39.609 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:01:39 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b[66141]: 2024-09-10T10:01:39.471+0000 7f349d53b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:01:39.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:39 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1269752429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 -- 172.21.15.45:0/480530980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4102790 msgr2=0x7f38c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 --2- 172.21.15.45:0/480530980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4102790 0x7f38c41067e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b002a410 comp rx=0 tx=0).stop 2024-09-10T10:01:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 -- 172.21.15.45:0/480530980 shutdown_connections 2024-09-10T10:01:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 --2- 172.21.15.45:0/480530980 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38c4106f10 0x7f38c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:39.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 --2- 172.21.15.45:0/480530980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4102790 0x7f38c41067e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 --2- 172.21.15.45:0/480530980 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38c4101de0 0x7f38c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 -- 172.21.15.45:0/480530980 >> 172.21.15.45:0/480530980 conn(0x7f38c40fd8c0 msgr2=0x7f38c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 -- 172.21.15.45:0/480530980 shutdown_connections 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.750+0000 7f38cad85640 1 -- 172.21.15.45:0/480530980 wait complete. 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 Processor -- start 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 -- start start 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4101de0 0x7f38c419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38c4102790 0x7f38c419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38c4106f10 0x7f38c419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4111f90 con 0x7f38c4101de0 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4111e10 con 0x7f38c4106f10 2024-09-10T10:01:39.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38cad85640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4112110 con 0x7f38c4102790 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4101de0 0x7f38c419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4101de0 0x7f38c419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41310/0 (socket says 172.21.15.45:41310) 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 -- 172.21.15.45:0/1814887240 learned_addr learned my addr 172.21.15.45:0/1814887240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c9322640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38c4106f10 0x7f38c419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c3fff640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38c4102790 0x7f38c419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 -- 172.21.15.45:0/1814887240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38c4102790 msgr2=0x7f38c419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38c4102790 0x7f38c419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 -- 172.21.15.45:0/1814887240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38c4106f10 msgr2=0x7f38c419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38c4106f10 0x7f38c419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.751+0000 7f38c8b21640 1 -- 172.21.15.45:0/1814887240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c419f640 con 0x7f38c4101de0 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.752+0000 7f38c8b21640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4101de0 0x7f38c419a690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38b800b4a0 tx=0x7f38b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.752+0000 7f38c1ffb640 1 -- 172.21.15.45:0/1814887240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8014070 con 0x7f38c4101de0 2024-09-10T10:01:39.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.752+0000 7f38c1ffb640 1 -- 172.21.15.45:0/1814887240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b8010af0 con 0x7f38c4101de0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.752+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c419f930 con 0x7f38c4101de0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.752+0000 7f38c1ffb640 1 -- 172.21.15.45:0/1814887240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8017d10 con 0x7f38c4101de0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.752+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41a7210 con 0x7f38c4101de0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.753+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c41021c0 con 0x7f38c4101de0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.754+0000 7f38c1ffb640 1 -- 172.21.15.45:0/1814887240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f38b8017070 con 0x7f38c4101de0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.755+0000 7f38c1ffb640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38b40a2b80 0x7f38b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.755+0000 7f38c3fff640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38b40a2b80 0x7f38b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.755+0000 7f38c1ffb640 1 -- 172.21.15.45:0/1814887240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f38b80ce2c0 con 0x7f38c4101de0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.755+0000 7f38c3fff640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38b40a2b80 0x7f38b40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38c40ff750 tx=0x7f38b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:39.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.759+0000 7f38c1ffb640 1 -- 172.21.15.45:0/1814887240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38b809b370 con 0x7f38c4101de0 2024-09-10T10:01:39.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:39.976+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f38c41a7440 con 0x7f38c4101de0 2024-09-10T10:01:40.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.254+0000 7f38c1ffb640 1 -- 172.21.15.45:0/1814887240 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f38b809ac70 con 0x7f38c4101de0 2024-09-10T10:01:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38b40a2b80 msgr2=0x7f38b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38b40a2b80 0x7f38b40a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38c40ff750 tx=0x7f38b0005a20 comp rx=0 tx=0).stop 2024-09-10T10:01:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4101de0 msgr2=0x7f38c419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4101de0 0x7f38c419a690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38b800b4a0 tx=0x7f38b800b960 comp rx=0 tx=0).stop 2024-09-10T10:01:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 shutdown_connections 2024-09-10T10:01:40.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f38b40a2b80 0x7f38b40a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38c4106f10 0x7f38c419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38c4102790 0x7f38c419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 --2- 172.21.15.45:0/1814887240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38c4101de0 0x7f38c419a690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 >> 172.21.15.45:0/1814887240 conn(0x7f38c40fd8c0 msgr2=0x7f38c40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 shutdown_connections 2024-09-10T10:01:40.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:40.257+0000 7f38cad85640 1 -- 172.21.15.45:0/1814887240 wait complete. 2024-09-10T10:01:40.628 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:40 smithi045 ceph-mon[31114]: pgmap v276: 97 pgs: 2 creating+peering, 95 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-10T10:01:40.629 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:40 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1814887240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:01:40.643 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:40 smithi162 ceph-mon[34963]: pgmap v276: 97 pgs: 2 creating+peering, 95 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-10T10:01:40.644 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:40 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1814887240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:01:40.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:40 smithi047 ceph-mon[35244]: pgmap v276: 97 pgs: 2 creating+peering, 95 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-10T10:01:40.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:40 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1814887240' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T10:01:40.918 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-10T10:01:40.918 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T10:01:41.176 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:41.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:41 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1814887240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:01:41.559 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:41 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:41.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:41 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1814887240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:01:41.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:41 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:41.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:41 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1814887240' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T10:01:41.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:41 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:41.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 -- 172.21.15.45:0/3097111905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4106050 msgr2=0x7f56d410dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 --2- 172.21.15.45:0/3097111905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4106050 0x7f56d410dbd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009ca0 tx=0x7f56cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 -- 172.21.15.45:0/3097111905 shutdown_connections 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 --2- 172.21.15.45:0/3097111905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4106050 0x7f56d410dbd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 --2- 172.21.15.45:0/3097111905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d4101df0 0x7f56d4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 --2- 172.21.15.45:0/3097111905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d4101440 0x7f56d4101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 -- 172.21.15.45:0/3097111905 >> 172.21.15.45:0/3097111905 conn(0x7f56d4077210 msgr2=0x7f56d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.642+0000 7f56d97cf640 1 -- 172.21.15.45:0/3097111905 shutdown_connections 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.643+0000 7f56d97cf640 1 -- 172.21.15.45:0/3097111905 wait complete. 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.643+0000 7f56d97cf640 1 Processor -- start 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.643+0000 7f56d97cf640 1 -- start start 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d97cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d4101440 0x7f56d41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d97cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4101df0 0x7f56d4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d97cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d4106050 0x7f56d419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d97cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4111070 con 0x7f56d4101df0 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d97cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4110ef0 con 0x7f56d4106050 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d97cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41111f0 con 0x7f56d4101440 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4101df0 0x7f56d4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4101df0 0x7f56d4198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41326/0 (socket says 172.21.15.45:41326) 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 -- 172.21.15.45:0/1255810161 learned_addr learned my addr 172.21.15.45:0/1255810161 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d2ffd640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d4101440 0x7f56d41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d37fe640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d4106050 0x7f56d419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 -- 172.21.15.45:0/1255810161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d4101440 msgr2=0x7f56d41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d4101440 0x7f56d41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 -- 172.21.15.45:0/1255810161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d4106050 msgr2=0x7f56d419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:41.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d4106050 0x7f56d419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 -- 172.21.15.45:0/1255810161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d419d580 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.644+0000 7f56d27fc640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4101df0 0x7f56d4198b10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f56c800c920 tx=0x7f56c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.645+0000 7f56bbfff640 1 -- 172.21.15.45:0/1255810161 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8007c70 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.645+0000 7f56bbfff640 1 -- 172.21.15.45:0/1255810161 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56c8007e10 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.645+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d419d870 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.645+0000 7f56bbfff640 1 -- 172.21.15.45:0/1255810161 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8018890 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.645+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d41a5150 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.646+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d4101820 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.647+0000 7f56bbfff640 1 -- 172.21.15.45:0/1255810161 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f56c8018a70 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.647+0000 7f56bbfff640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f56bc0a2b80 0x7f56bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.648+0000 7f56d2ffd640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f56bc0a2b80 0x7f56bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.648+0000 7f56bbfff640 1 -- 172.21.15.45:0/1255810161 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f56c80cf090 con 0x7f56d4101df0 2024-09-10T10:01:41.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.648+0000 7f56d2ffd640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f56bc0a2b80 0x7f56bc0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005fd0 tx=0x7f56c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:41.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.652+0000 7f56bbfff640 1 -- 172.21.15.45:0/1255810161 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56c8014030 con 0x7f56d4101df0 2024-09-10T10:01:41.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:41.867+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f56d41a5320 con 0x7f56d4101df0 2024-09-10T10:01:42.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.260+0000 7f56bbfff640 1 -- 172.21.15.45:0/1255810161 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f56c809c0a0 con 0x7f56d4101df0 2024-09-10T10:01:42.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f56bc0a2b80 msgr2=0x7f56bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f56bc0a2b80 0x7f56bc0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005fd0 tx=0x7f56c400b390 comp rx=0 tx=0).stop 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4101df0 msgr2=0x7f56d4198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4101df0 0x7f56d4198b10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f56c800c920 tx=0x7f56c800cde0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 shutdown_connections 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f56bc0a2b80 0x7f56bc0a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56d4106050 0x7f56d419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d4101df0 0x7f56d4198b10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 --2- 172.21.15.45:0/1255810161 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56d4101440 0x7f56d41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 >> 172.21.15.45:0/1255810161 conn(0x7f56d4077210 msgr2=0x7f56d41001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 shutdown_connections 2024-09-10T10:01:42.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:42.263+0000 7f56d97cf640 1 -- 172.21.15.45:0/1255810161 wait complete. 2024-09-10T10:01:42.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:42 smithi045 ceph-mon[31114]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-10T10:01:42.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:42 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1255810161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:01:42.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:42 smithi162 ceph-mon[34963]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-10T10:01:42.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:42 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1255810161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:01:42.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:42 smithi047 ceph-mon[35244]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-10T10:01:42.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:42 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1255810161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T10:01:42.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T10:01:43.226 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:43.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:43 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1255810161' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:01:43.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:43 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:43.625 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:43 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:01:43.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:43 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1255810161' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:01:43.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:43 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:43.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:43 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:01:43.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:43 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1255810161' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T10:01:43.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:43 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:01:43.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:43 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 -- 172.21.15.45:0/3398543524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8106a60 msgr2=0x7f98a8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/3398543524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8106a60 0x7f98a8106e40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9898009930 tx=0x7f989802a420 comp rx=0 tx=0).stop 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 -- 172.21.15.45:0/3398543524 shutdown_connections 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/3398543524 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a8101400 0x7f98a810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/3398543524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a8100a60 0x7f98a8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/3398543524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8106a60 0x7f98a8106e40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 -- 172.21.15.45:0/3398543524 >> 172.21.15.45:0/3398543524 conn(0x7f98a80fc780 msgr2=0x7f98a80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 -- 172.21.15.45:0/3398543524 shutdown_connections 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.708+0000 7f98ac9dc640 1 -- 172.21.15.45:0/3398543524 wait complete. 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 Processor -- start 2024-09-10T10:01:43.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 -- start start 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a8100a60 0x7f98a8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8101400 0x7f98a81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a8106a60 0x7f98a81a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8110e60 con 0x7f98a8101400 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8110ce0 con 0x7f98a8100a60 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98ac9dc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f98a8110fe0 con 0x7f98a8106a60 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8101400 0x7f98a81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8101400 0x7f98a81997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43542/0 (socket says 172.21.15.45:43542) 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 -- 172.21.15.45:0/2483836247 learned_addr learned my addr 172.21.15.45:0/2483836247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98a659c640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a8100a60 0x7f98a8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98a6d9d640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a8106a60 0x7f98a81a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 -- 172.21.15.45:0/2483836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a8106a60 msgr2=0x7f98a81a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a8106a60 0x7f98a81a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 -- 172.21.15.45:0/2483836247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a8100a60 msgr2=0x7f98a8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a8100a60 0x7f98a8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f989ffff640 1 -- 172.21.15.45:0/2483836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a81a4830 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.709+0000 7f98a6d9d640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a8106a60 0x7f98a81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.710+0000 7f989ffff640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8101400 0x7f98a81997b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f989000d950 tx=0x7f989000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.710+0000 7f989f7fe640 1 -- 172.21.15.45:0/2483836247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890017070 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.710+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a81a4a00 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.710+0000 7f989f7fe640 1 -- 172.21.15.45:0/2483836247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9890009c30 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.710+0000 7f989f7fe640 1 -- 172.21.15.45:0/2483836247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989000f6b0 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.710+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a81a4ef0 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.711+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a8100ec0 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.712+0000 7f989f7fe640 1 -- 172.21.15.45:0/2483836247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f98900109d0 con 0x7f98a8101400 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.713+0000 7f989f7fe640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98940a2b80 0x7f98940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:43.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.713+0000 7f98a659c640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98940a2b80 0x7f98940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.713+0000 7f989f7fe640 1 -- 172.21.15.45:0/2483836247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f98900ceb60 con 0x7f98a8101400 2024-09-10T10:01:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.713+0000 7f98a659c640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98940a2b80 0x7f98940a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9898009900 tx=0x7f98980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:43.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.717+0000 7f989f7fe640 1 -- 172.21.15.45:0/2483836247 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9890009de0 con 0x7f98a8101400 2024-09-10T10:01:43.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:43.931+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f98a8108fd0 con 0x7f98a8101400 2024-09-10T10:01:44.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.272+0000 7f989f7fe640 1 -- 172.21.15.45:0/2483836247 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f989009bc10 con 0x7f98a8101400 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98940a2b80 msgr2=0x7f98940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98940a2b80 0x7f98940a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9898009900 tx=0x7f98980058d0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8101400 msgr2=0x7f98a81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8101400 0x7f98a81997b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f989000d950 tx=0x7f989000de10 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 shutdown_connections 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f98940a2b80 0x7f98940a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f98a8106a60 0x7f98a81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98a8101400 0x7f98a81997b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 --2- 172.21.15.45:0/2483836247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98a8100a60 0x7f98a8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:44.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 >> 172.21.15.45:0/2483836247 conn(0x7f98a80fc780 msgr2=0x7f98a80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 shutdown_connections 2024-09-10T10:01:44.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:44.275+0000 7f98ac9dc640 1 -- 172.21.15.45:0/2483836247 wait complete. 2024-09-10T10:01:44.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:44 smithi045 ceph-mon[31114]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 244 B/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-10T10:01:44.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:44 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2483836247' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:01:44.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:44 smithi162 ceph-mon[34963]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 244 B/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-10T10:01:44.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:44 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2483836247' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:01:44.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:44 smithi047 ceph-mon[35244]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 244 B/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-10T10:01:44.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:44 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2483836247' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T10:01:44.972 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-10T10:01:44.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T10:01:45.233 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:45.575 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:45 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2483836247' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:01:45.575 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:45 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:01:45.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:45 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2483836247' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:01:45.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:45 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:01:45.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:45 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2483836247' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T10:01:45.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:45 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:01:45.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 -- 172.21.15.45:0/1862829197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37081082e0 msgr2=0x7f370810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 --2- 172.21.15.45:0/1862829197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37081082e0 0x7f370810ec50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3700009930 tx=0x7f370002a420 comp rx=0 tx=0).stop 2024-09-10T10:01:45.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 -- 172.21.15.45:0/1862829197 shutdown_connections 2024-09-10T10:01:45.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 --2- 172.21.15.45:0/1862829197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37081082e0 0x7f370810ec50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 --2- 172.21.15.45:0/1862829197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3708103760 0x7f3708107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 --2- 172.21.15.45:0/1862829197 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3708102db0 0x7f3708103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 -- 172.21.15.45:0/1862829197 >> 172.21.15.45:0/1862829197 conn(0x7f37080fc7d0 msgr2=0x7f37080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 -- 172.21.15.45:0/1862829197 shutdown_connections 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 -- 172.21.15.45:0/1862829197 wait complete. 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.659+0000 7f370e199640 1 Processor -- start 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f370e199640 1 -- start start 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f370e199640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3708102db0 0x7f37081a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f370e199640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3708103760 0x7f37081a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f370e199640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37081082e0 0x7f370819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f370e199640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37081120b0 con 0x7f3708103760 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f370e199640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3708111f30 con 0x7f3708102db0 2024-09-10T10:01:45.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f370e199640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3708112230 con 0x7f37081082e0 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3708103760 0x7f37081a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3708103760 0x7f37081a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43572/0 (socket says 172.21.15.45:43572) 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 -- 172.21.15.45:0/2276049401 learned_addr learned my addr 172.21.15.45:0/2276049401 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3707fff640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37081082e0 0x7f370819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f37077fe640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3708102db0 0x7f37081a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 -- 172.21.15.45:0/2276049401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37081082e0 msgr2=0x7f370819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37081082e0 0x7f370819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 -- 172.21.15.45:0/2276049401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3708102db0 msgr2=0x7f37081a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3708102db0 0x7f37081a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 -- 172.21.15.45:0/2276049401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370819aed0 con 0x7f3708103760 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3707fff640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37081082e0 0x7f370819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.660+0000 7f3706ffd640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3708103760 0x7f37081a0ad0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d740 tx=0x7f36f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:45.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.661+0000 7f3704ff9640 1 -- 172.21.15.45:0/2276049401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0017070 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.661+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370819b1c0 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.661+0000 7f3704ff9640 1 -- 172.21.15.45:0/2276049401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f000de50 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.661+0000 7f3704ff9640 1 -- 172.21.15.45:0/2276049401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f000f6b0 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.661+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081a7510 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.662+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370806a7d0 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.663+0000 7f3704ff9640 1 -- 172.21.15.45:0/2276049401 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f36f001a070 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.664+0000 7f3704ff9640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36f40a2b80 0x7f36f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.664+0000 7f37077fe640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36f40a2b80 0x7f36f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.664+0000 7f3704ff9640 1 -- 172.21.15.45:0/2276049401 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f36f00cea50 con 0x7f3708103760 2024-09-10T10:01:45.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.664+0000 7f37077fe640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36f40a2b80 0x7f36f40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005fd0 tx=0x7f36f8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:45.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.668+0000 7f3704ff9640 1 -- 172.21.15.45:0/2276049401 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36f009ba60 con 0x7f3708103760 2024-09-10T10:01:45.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.867+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f370819bb90 con 0x7f36f40a2b80 2024-09-10T10:01:45.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.888+0000 7f3704ff9640 1 -- 172.21.15.45:0/2276049401 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f370819bb90 con 0x7f36f40a2b80 2024-09-10T10:01:45.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36f40a2b80 msgr2=0x7f36f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36f40a2b80 0x7f36f40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005fd0 tx=0x7f36f8009500 comp rx=0 tx=0).stop 2024-09-10T10:01:45.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3708103760 msgr2=0x7f37081a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:45.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3708103760 0x7f37081a0ad0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36f000d740 tx=0x7f36f000dc00 comp rx=0 tx=0).stop 2024-09-10T10:01:45.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 shutdown_connections 2024-09-10T10:01:45.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f36f40a2b80 0x7f36f40a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37081082e0 0x7f370819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3708103760 0x7f37081a0ad0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 --2- 172.21.15.45:0/2276049401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3708102db0 0x7f37081a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:45.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 >> 172.21.15.45:0/2276049401 conn(0x7f37080fc7d0 msgr2=0x7f37080fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:45.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 shutdown_connections 2024-09-10T10:01:45.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:45.890+0000 7f370e199640 1 -- 172.21.15.45:0/2276049401 wait complete. 2024-09-10T10:01:46.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:01:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:01:45.873+0000 7ff66ae3d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:46.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:01:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:01:45.873+0000 7ff66ae3d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:46.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:01:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:01:45.873+0000 7ff66ae3d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:46.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:01:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:01:45.873+0000 7ff66ae3d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:46.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:01:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:01:45.873+0000 7ff66ae3d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:01:46.488 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:46 smithi045 ceph-mon[31114]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T10:01:46.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T10:01:46.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:46 smithi162 ceph-mon[34963]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T10:01:46.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:46 smithi047 ceph-mon[35244]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-10T10:01:46.784 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 -- 172.21.15.45:0/676301753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 msgr2=0x7f3954101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 --2- 172.21.15.45:0/676301753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f3954101200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3944009ca0 tx=0x7f394402a950 comp rx=0 tx=0).stop 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 -- 172.21.15.45:0/676301753 shutdown_connections 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 --2- 172.21.15.45:0/676301753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3954102ef0 0x7f395410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 --2- 172.21.15.45:0/676301753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3954102570 0x7f39541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 --2- 172.21.15.45:0/676301753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f3954101200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 -- 172.21.15.45:0/676301753 >> 172.21.15.45:0/676301753 conn(0x7f39540fc7d0 msgr2=0x7f39540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 -- 172.21.15.45:0/676301753 shutdown_connections 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.311+0000 7f395b393640 1 -- 172.21.15.45:0/676301753 wait complete. 2024-09-10T10:01:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 Processor -- start 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 -- start start 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f39541970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3954102570 0x7f3954197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3954102ef0 0x7f3954197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3954110f80 con 0x7f3954100e20 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3954110e00 con 0x7f3954102570 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395b393640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3954111100 con 0x7f3954102ef0 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f39541970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f39541970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43610/0 (socket says 172.21.15.45:43610) 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 -- 172.21.15.45:0/2622132213 learned_addr learned my addr 172.21.15.45:0/2622132213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:47.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395892e640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3954102570 0x7f3954197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f3959930640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3954102ef0 0x7f3954197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 -- 172.21.15.45:0/2622132213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3954102ef0 msgr2=0x7f3954197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3954102ef0 0x7f3954197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 -- 172.21.15.45:0/2622132213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3954102570 msgr2=0x7f3954197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3954102570 0x7f3954197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.312+0000 7f395912f640 1 -- 172.21.15.45:0/2622132213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39541a6950 con 0x7f3954100e20 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.313+0000 7f395912f640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f39541970e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3944005cc0 tx=0x7f394402f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.313+0000 7f39527fc640 1 -- 172.21.15.45:0/2622132213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944034040 con 0x7f3954100e20 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.313+0000 7f39527fc640 1 -- 172.21.15.45:0/2622132213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3944030920 con 0x7f3954100e20 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.313+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39541a6be0 con 0x7f3954100e20 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.313+0000 7f39527fc640 1 -- 172.21.15.45:0/2622132213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944004830 con 0x7f3954100e20 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.313+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541a7110 con 0x7f3954100e20 2024-09-10T10:01:47.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.314+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3954101200 con 0x7f3954100e20 2024-09-10T10:01:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.315+0000 7f39527fc640 1 -- 172.21.15.45:0/2622132213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3944037020 con 0x7f3954100e20 2024-09-10T10:01:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.316+0000 7f39527fc640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f393c0a2c90 0x7f393c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.316+0000 7f39527fc640 1 -- 172.21.15.45:0/2622132213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f39440efef0 con 0x7f3954100e20 2024-09-10T10:01:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.316+0000 7f395892e640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f393c0a2c90 0x7f393c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.317+0000 7f395892e640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f393c0a2c90 0x7f393c0a5150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3954199240 tx=0x7f394800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:47.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.321+0000 7f39527fc640 1 -- 172.21.15.45:0/2622132213 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39440bcfb0 con 0x7f3954100e20 2024-09-10T10:01:47.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:47 smithi045 ceph-mon[31114]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:47.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.521+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f39541090c0 con 0x7f393c0a2c90 2024-09-10T10:01:47.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.545+0000 7f39527fc640 1 -- 172.21.15.45:0/2622132213 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39541090c0 con 0x7f393c0a2c90 2024-09-10T10:01:47.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f393c0a2c90 msgr2=0x7f393c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f393c0a2c90 0x7f393c0a5150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3954199240 tx=0x7f394800b390 comp rx=0 tx=0).stop 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 msgr2=0x7f39541970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f39541970e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3944005cc0 tx=0x7f394402f650 comp rx=0 tx=0).stop 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 shutdown_connections 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f393c0a2c90 0x7f393c0a5150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3954102ef0 0x7f3954197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3954102570 0x7f3954197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 --2- 172.21.15.45:0/2622132213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954100e20 0x7f39541970e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 >> 172.21.15.45:0/2622132213 conn(0x7f39540fc7d0 msgr2=0x7f39540fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 shutdown_connections 2024-09-10T10:01:47.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:47.547+0000 7f395b393640 1 -- 172.21.15.45:0/2622132213 wait complete. 2024-09-10T10:01:47.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:47 smithi162 ceph-mon[34963]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:47 smithi047 ceph-mon[35244]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:48.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T10:01:48.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:48 smithi045 ceph-mon[31114]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T10:01:48.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:48 smithi045 ceph-mon[31114]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-09-10T10:01:48.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:48 smithi045 ceph-mon[31114]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:48.427 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:48.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:48 smithi162 ceph-mon[34963]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T10:01:48.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:48 smithi162 ceph-mon[34963]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-09-10T10:01:48.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:48 smithi162 ceph-mon[34963]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:48 smithi047 ceph-mon[35244]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T10:01:48.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:48 smithi047 ceph-mon[35244]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-09-10T10:01:48.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:48 smithi047 ceph-mon[35244]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:48.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 -- 172.21.15.45:0/381916034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c106a80 msgr2=0x7fde9c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 --2- 172.21.15.45:0/381916034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c106a80 0x7fde9c106e60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009930 tx=0x7fde8c02a420 comp rx=0 tx=0).stop 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 -- 172.21.15.45:0/381916034 shutdown_connections 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 --2- 172.21.15.45:0/381916034 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde9c101420 0x7fde9c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 --2- 172.21.15.45:0/381916034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c100a80 0x7fde9c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 --2- 172.21.15.45:0/381916034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c106a80 0x7fde9c106e60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 -- 172.21.15.45:0/381916034 >> 172.21.15.45:0/381916034 conn(0x7fde9c0fc7a0 msgr2=0x7fde9c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 -- 172.21.15.45:0/381916034 shutdown_connections 2024-09-10T10:01:48.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.885+0000 7fdea0a15640 1 -- 172.21.15.45:0/381916034 wait complete. 2024-09-10T10:01:48.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 Processor -- start 2024-09-10T10:01:48.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 -- start start 2024-09-10T10:01:48.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c100a80 0x7fde9c0fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde9c101420 0x7fde9c0fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c106a80 0x7fde9c0fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c110e90 con 0x7fde9c100a80 2024-09-10T10:01:48.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c110d10 con 0x7fde9c106a80 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.886+0000 7fdea0a15640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c111010 con 0x7fde9c101420 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c100a80 0x7fde9c0fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c100a80 0x7fde9c0fcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43624/0 (socket says 172.21.15.45:43624) 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 -- 172.21.15.45:0/357371234 learned_addr learned my addr 172.21.15.45:0/357371234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde93fff640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde9c101420 0x7fde9c0fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9ad9d640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c106a80 0x7fde9c0fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 -- 172.21.15.45:0/357371234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde9c101420 msgr2=0x7fde9c0fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde9c101420 0x7fde9c0fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 -- 172.21.15.45:0/357371234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c106a80 msgr2=0x7fde9c0fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c106a80 0x7fde9c0fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 -- 172.21.15.45:0/357371234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c1a6950 con 0x7fde9c100a80 2024-09-10T10:01:48.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde9a59c640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c100a80 0x7fde9c0fcdf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fde8c02a940 tx=0x7fde8c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde937fe640 1 -- 172.21.15.45:0/357371234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c03f070 con 0x7fde9c100a80 2024-09-10T10:01:48.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde937fe640 1 -- 172.21.15.45:0/357371234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c03b460 con 0x7fde9c100a80 2024-09-10T10:01:48.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c1a6be0 con 0x7fde9c100a80 2024-09-10T10:01:48.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fde937fe640 1 -- 172.21.15.45:0/357371234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c035030 con 0x7fde9c100a80 2024-09-10T10:01:48.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.887+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c1a70c0 con 0x7fde9c100a80 2024-09-10T10:01:48.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.889+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c100ee0 con 0x7fde9c100a80 2024-09-10T10:01:48.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.890+0000 7fde937fe640 1 -- 172.21.15.45:0/357371234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fde8c042070 con 0x7fde9c100a80 2024-09-10T10:01:48.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.890+0000 7fde937fe640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fde880a2c10 0x7fde880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:48.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.890+0000 7fde93fff640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fde880a2c10 0x7fde880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:48.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.890+0000 7fde937fe640 1 -- 172.21.15.45:0/357371234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fde8c0ef980 con 0x7fde9c100a80 2024-09-10T10:01:48.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.891+0000 7fde93fff640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fde880a2c10 0x7fde880a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0fe880 tx=0x7fde8400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:48.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:48.895+0000 7fde937fe640 1 -- 172.21.15.45:0/357371234 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde8c0bc930 con 0x7fde9c100a80 2024-09-10T10:01:49.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.098+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fde9c10a070 con 0x7fde880a2c10 2024-09-10T10:01:49.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.103+0000 7fde937fe640 1 -- 172.21.15.45:0/357371234 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fde8400ea60 con 0x7fde880a2c10 2024-09-10T10:01:49.105 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_0/c3f3b371-2e02-42e9-a1fe-8145c5150a01 2024-09-10T10:01:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fde880a2c10 msgr2=0x7fde880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fde880a2c10 0x7fde880a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0fe880 tx=0x7fde8400a4c0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c100a80 msgr2=0x7fde9c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c100a80 0x7fde9c0fcdf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fde8c02a940 tx=0x7fde8c003c00 comp rx=0 tx=0).stop 2024-09-10T10:01:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 shutdown_connections 2024-09-10T10:01:49.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fde880a2c10 0x7fde880a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde9c106a80 0x7fde9c0fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fde9c101420 0x7fde9c0fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 --2- 172.21.15.45:0/357371234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde9c100a80 0x7fde9c0fcdf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 >> 172.21.15.45:0/357371234 conn(0x7fde9c0fc7a0 msgr2=0x7fde9c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 shutdown_connections 2024-09-10T10:01:49.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:49.105+0000 7fdea0a15640 1 -- 172.21.15.45:0/357371234 wait complete. 2024-09-10T10:01:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T10:01:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-10T10:01:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-09-10T10:01:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-10T10:01:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-10T10:01:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-09-10T10:01:49.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T10:01:49.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:49 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-09-10T10:01:49.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T10:01:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-10T10:01:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-09-10T10:01:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-10T10:01:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-10T10:01:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-09-10T10:01:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T10:01:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:49 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T10:01:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:49 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-09-10T10:01:49.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T10:01:49.988 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:50.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: pgmap v281: 97 pgs: 97 active+clean; 585 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-10T10:01:50.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:50.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T10:01:50.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T10:01:50.391 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-10T10:01:50.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-10T10:01:50.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-10T10:01:50.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-10T10:01:50.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T10:01:50.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-10T10:01:50.392 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:50 smithi045 ceph-mon[31114]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.474+0000 7f93810f2640 1 -- 172.21.15.45:0/1725096655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 msgr2=0x7f937c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.474+0000 7f93810f2640 1 --2- 172.21.15.45:0/1725096655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c103110 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f936c009cd0 tx=0x7f936c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.474+0000 7f93810f2640 1 -- 172.21.15.45:0/1725096655 shutdown_connections 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.474+0000 7f93810f2640 1 --2- 172.21.15.45:0/1725096655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f937c1082a0 0x7f937c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.474+0000 7f93810f2640 1 --2- 172.21.15.45:0/1725096655 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f937c1036e0 0x7f937c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.474+0000 7f93810f2640 1 --2- 172.21.15.45:0/1725096655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c103110 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.474+0000 7f93810f2640 1 -- 172.21.15.45:0/1725096655 >> 172.21.15.45:0/1725096655 conn(0x7f937c0fc7d0 msgr2=0x7f937c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.475+0000 7f93810f2640 1 -- 172.21.15.45:0/1725096655 shutdown_connections 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.475+0000 7f93810f2640 1 -- 172.21.15.45:0/1725096655 wait complete. 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.475+0000 7f93810f2640 1 Processor -- start 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f93810f2640 1 -- start start 2024-09-10T10:01:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f93810f2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f93810f2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f937c1036e0 0x7f937c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f93810f2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f937c1082a0 0x7f937c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f93810f2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f937c1120a0 con 0x7f937c102d30 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f93810f2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f937c111f20 con 0x7f937c1036e0 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f93810f2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f937c112220 con 0x7f937c1082a0 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43636/0 (socket says 172.21.15.45:43636) 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 -- 172.21.15.45:0/2711833452 learned_addr learned my addr 172.21.15.45:0/2711833452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937a59c640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f937c1036e0 0x7f937c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937b59e640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f937c1082a0 0x7f937c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 -- 172.21.15.45:0/2711833452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f937c1082a0 msgr2=0x7f937c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f937c1082a0 0x7f937c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 -- 172.21.15.45:0/2711833452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f937c1036e0 msgr2=0x7f937c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f937c1036e0 0x7f937c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937ad9d640 1 -- 172.21.15.45:0/2711833452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c19d4d0 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.476+0000 7f937b59e640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f937c1082a0 0x7f937c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.477+0000 7f937ad9d640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c198520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f936c02ae00 tx=0x7f936c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.477+0000 7f9363fff640 1 -- 172.21.15.45:0/2711833452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c004240 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.477+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c19d760 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.477+0000 7f9363fff640 1 -- 172.21.15.45:0/2711833452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f936c0305a0 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.477+0000 7f9363fff640 1 -- 172.21.15.45:0/2711833452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c03a660 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.477+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c1a50a0 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.478+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c103110 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.479+0000 7f9363fff640 1 -- 172.21.15.45:0/2711833452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f936c030ac0 con 0x7f937c102d30 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.479+0000 7f9363fff640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f93640a2c10 0x7f93640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.480+0000 7f937a59c640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f93640a2c10 0x7f93640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.480+0000 7f9363fff640 1 -- 172.21.15.45:0/2711833452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f936c02e030 con 0x7f937c102d30 2024-09-10T10:01:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.480+0000 7f937a59c640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f93640a2c10 0x7f93640a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f937c0fee50 tx=0x7f937000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:50.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.484+0000 7f9363fff640 1 -- 172.21.15.45:0/2711833452 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f936c0098b0 con 0x7f937c102d30 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: pgmap v281: 97 pgs: 97 active+clean; 585 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-10T10:01:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-10T10:01:50.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T10:01:50.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-10T10:01:50.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:50 smithi162 ceph-mon[34963]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:01:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: pgmap v281: 97 pgs: 97 active+clean; 585 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-10T10:01:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-10T10:01:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T10:01:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-10T10:01:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-10T10:01:50.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-10T10:01:50.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 7]}]': finished 2024-09-10T10:01:50.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T10:01:50.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-10T10:01:50.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:50 smithi047 ceph-mon[35244]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T10:01:50.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.686+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f937c107420 con 0x7f93640a2c10 2024-09-10T10:01:50.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.711+0000 7f9363fff640 1 -- 172.21.15.45:0/2711833452 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f937c107420 con 0x7f93640a2c10 2024-09-10T10:01:50.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f93640a2c10 msgr2=0x7f93640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f93640a2c10 0x7f93640a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f937c0fee50 tx=0x7f937000a2d0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 msgr2=0x7f937c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c198520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f936c02ae00 tx=0x7f936c003ba0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 shutdown_connections 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f93640a2c10 0x7f93640a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f937c1082a0 0x7f937c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f937c1036e0 0x7f937c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 --2- 172.21.15.45:0/2711833452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f937c102d30 0x7f937c198520 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 >> 172.21.15.45:0/2711833452 conn(0x7f937c0fc7d0 msgr2=0x7f937c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 shutdown_connections 2024-09-10T10:01:50.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:50.713+0000 7f93810f2640 1 -- 172.21.15.45:0/2711833452 wait complete. 2024-09-10T10:01:51.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T10:01:51.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:51 smithi045 ceph-mon[31114]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:01:51.639 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:51.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:51 smithi162 ceph-mon[34963]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:01:51.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:51 smithi047 ceph-mon[35244]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T10:01:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 -- 172.21.15.45:0/3555842035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0103660 msgr2=0x7faff010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 --2- 172.21.15.45:0/3555842035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0103660 0x7faff010bb50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009900 tx=0x7fafe002a480 comp rx=0 tx=0).stop 2024-09-10T10:01:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 -- 172.21.15.45:0/3555842035 shutdown_connections 2024-09-10T10:01:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 --2- 172.21.15.45:0/3555842035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff010c170 0x7faff010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 --2- 172.21.15.45:0/3555842035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0103660 0x7faff010bb50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 --2- 172.21.15.45:0/3555842035 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faff0102d40 0x7faff0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 -- 172.21.15.45:0/3555842035 >> 172.21.15.45:0/3555842035 conn(0x7faff00fc7a0 msgr2=0x7faff00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.126+0000 7faff77c2640 1 -- 172.21.15.45:0/3555842035 shutdown_connections 2024-09-10T10:01:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 -- 172.21.15.45:0/3555842035 wait complete. 2024-09-10T10:01:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 Processor -- start 2024-09-10T10:01:52.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 -- start start 2024-09-10T10:01:52.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0102d40 0x7faff019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff0103660 0x7faff019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faff010c170 0x7faff019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111fc0 con 0x7faff0102d40 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faff0111e40 con 0x7faff0103660 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff77c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7faff0112140 con 0x7faff010c170 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.127+0000 7faff555e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0102d40 0x7faff019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0102d40 0x7faff019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43662/0 (socket says 172.21.15.45:43662) 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 -- 172.21.15.45:0/566939234 learned_addr learned my addr 172.21.15.45:0/566939234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff4d5d640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff0103660 0x7faff019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff5d5f640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faff010c170 0x7faff019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 -- 172.21.15.45:0/566939234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faff010c170 msgr2=0x7faff019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faff010c170 0x7faff019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 -- 172.21.15.45:0/566939234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff0103660 msgr2=0x7faff019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff0103660 0x7faff019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 -- 172.21.15.45:0/566939234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff019f630 con 0x7faff0102d40 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff555e640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0102d40 0x7faff019a680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fafec00c890 tx=0x7fafec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7fafe67fc640 1 -- 172.21.15.45:0/566939234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec004220 con 0x7faff0102d40 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff019f920 con 0x7faff0102d40 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7fafe67fc640 1 -- 172.21.15.45:0/566939234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafec0043c0 con 0x7faff0102d40 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7fafe67fc640 1 -- 172.21.15.45:0/566939234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec004a00 con 0x7faff0102d40 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.128+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff01a7200 con 0x7faff0102d40 2024-09-10T10:01:52.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.129+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff0103120 con 0x7faff0102d40 2024-09-10T10:01:52.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.131+0000 7fafe67fc640 1 -- 172.21.15.45:0/566939234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fafec0079b0 con 0x7faff0102d40 2024-09-10T10:01:52.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.131+0000 7fafe67fc640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fafdc0a2c10 0x7fafdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:52.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.132+0000 7faff4d5d640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fafdc0a2c10 0x7fafdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:52.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.132+0000 7fafe67fc640 1 -- 172.21.15.45:0/566939234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fafec0ceec0 con 0x7faff0102d40 2024-09-10T10:01:52.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.132+0000 7faff4d5d640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fafdc0a2c10 0x7fafdc0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fafe0000c00 tx=0x7fafe0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:52.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.136+0000 7fafe67fc640 1 -- 172.21.15.45:0/566939234 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafec014030 con 0x7faff0102d40 2024-09-10T10:01:52.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.339+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7faff00feac0 con 0x7fafdc0a2c10 2024-09-10T10:01:52.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.344+0000 7fafe67fc640 1 -- 172.21.15.45:0/566939234 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7faff00feac0 con 0x7fafdc0a2c10 2024-09-10T10:01:52.346 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_1/7f8e0794-462f-4a26-a474-c21ab28cee52 2024-09-10T10:01:52.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fafdc0a2c10 msgr2=0x7fafdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fafdc0a2c10 0x7fafdc0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fafe0000c00 tx=0x7fafe0005b50 comp rx=0 tx=0).stop 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0102d40 msgr2=0x7faff019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0102d40 0x7faff019a680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fafec00c890 tx=0x7fafec00cd50 comp rx=0 tx=0).stop 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 shutdown_connections 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fafdc0a2c10 0x7fafdc0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faff010c170 0x7faff019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faff0103660 0x7faff019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 --2- 172.21.15.45:0/566939234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faff0102d40 0x7faff019a680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 >> 172.21.15.45:0/566939234 conn(0x7faff00fc7a0 msgr2=0x7faff00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 shutdown_connections 2024-09-10T10:01:52.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:52.346+0000 7faff77c2640 1 -- 172.21.15.45:0/566939234 wait complete. 2024-09-10T10:01:52.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:52 smithi162 ceph-mon[34963]: pgmap v284: 97 pgs: 97 active+clean; 585 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:01:52.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:52 smithi162 ceph-mon[34963]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:52.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:52 smithi045 ceph-mon[31114]: pgmap v284: 97 pgs: 97 active+clean; 585 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:01:52.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:52 smithi045 ceph-mon[31114]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:52.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:52 smithi047 ceph-mon[35244]: pgmap v284: 97 pgs: 97 active+clean; 585 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:01:52.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:52 smithi047 ceph-mon[35244]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:52.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:53.253 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:53.624 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:53 smithi045 ceph-mon[31114]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:53.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:53 smithi162 ceph-mon[34963]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:53.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:53 smithi047 ceph-mon[35244]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:01:53.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.704+0000 7f30622c8640 1 -- 172.21.15.45:0/3842049319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 msgr2=0x7f305c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.704+0000 7f30622c8640 1 --2- 172.21.15.45:0/3842049319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c10ec30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3054009cd0 tx=0x7f305402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 -- 172.21.15.45:0/3842049319 shutdown_connections 2024-09-10T10:01:53.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 --2- 172.21.15.45:0/3842049319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c10ec30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 --2- 172.21.15.45:0/3842049319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c103740 0x7f305c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 --2- 172.21.15.45:0/3842049319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f305c102d90 0x7f305c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 -- 172.21.15.45:0/3842049319 >> 172.21.15.45:0/3842049319 conn(0x7f305c0fc7d0 msgr2=0x7f305c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 -- 172.21.15.45:0/3842049319 shutdown_connections 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 -- 172.21.15.45:0/3842049319 wait complete. 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 Processor -- start 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.705+0000 7f30622c8640 1 -- start start 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30622c8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c102d90 0x7f305c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30622c8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f305c103740 0x7f305c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30622c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30622c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1120d0 con 0x7f305c108290 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30622c8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f305c111f50 con 0x7f305c103740 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30622c8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f305c112250 con 0x7f305c102d90 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34796/0 (socket says 172.21.15.45:34796) 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 -- 172.21.15.45:0/511365612 learned_addr learned my addr 172.21.15.45:0/511365612 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f305bfff640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c102d90 0x7f305c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f305b7fe640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f305c103740 0x7f305c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 -- 172.21.15.45:0/511365612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c102d90 msgr2=0x7f305c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c102d90 0x7f305c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 -- 172.21.15.45:0/511365612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f305c103740 msgr2=0x7f305c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f305c103740 0x7f305c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 -- 172.21.15.45:0/511365612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c19d560 con 0x7f305c108290 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f3060865640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c19ce80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3054005cc0 tx=0x7f3054003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30597fa640 1 -- 172.21.15.45:0/511365612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054004040 con 0x7f305c108290 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.706+0000 7f30597fa640 1 -- 172.21.15.45:0/511365612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30540046c0 con 0x7f305c108290 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.707+0000 7f30597fa640 1 -- 172.21.15.45:0/511365612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054030e70 con 0x7f305c108290 2024-09-10T10:01:53.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.707+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c19d7f0 con 0x7f305c108290 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.707+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c1a5130 con 0x7f305c108290 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.708+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305c103170 con 0x7f305c108290 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.709+0000 7f30597fa640 1 -- 172.21.15.45:0/511365612 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f30540305a0 con 0x7f305c108290 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.709+0000 7f30597fa640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f30440a2c90 0x7f30440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.709+0000 7f305bfff640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f30440a2c90 0x7f30440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.709+0000 7f30597fa640 1 -- 172.21.15.45:0/511365612 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f305402e030 con 0x7f305c108290 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.710+0000 7f305bfff640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f30440a2c90 0x7f30440a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f304c00bbe0 tx=0x7f304c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:53.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.713+0000 7f30597fa640 1 -- 172.21.15.45:0/511365612 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30540bd020 con 0x7f305c108290 2024-09-10T10:01:53.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.929+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f305c10a2a0 con 0x7f305c108290 2024-09-10T10:01:53.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.930+0000 7f30597fa640 1 -- 172.21.15.45:0/511365612 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11843 (secure 0 0 0) 0x7f30540098b0 con 0x7f305c108290 2024-09-10T10:01:53.933 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:53.933 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"btime":"2024-09-10T10:01:44:271138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:01:43.932833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:01:53.933 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-09-10T10:01:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f30440a2c90 msgr2=0x7f30440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f30440a2c90 0x7f30440a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f304c00bbe0 tx=0x7f304c00b390 comp rx=0 tx=0).stop 2024-09-10T10:01:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 msgr2=0x7f305c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c19ce80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3054005cc0 tx=0x7f3054003970 comp rx=0 tx=0).stop 2024-09-10T10:01:53.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 shutdown_connections 2024-09-10T10:01:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f30440a2c90 0x7f30440a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f305c108290 0x7f305c19ce80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f305c103740 0x7f305c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 --2- 172.21.15.45:0/511365612 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f305c102d90 0x7f305c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 >> 172.21.15.45:0/511365612 conn(0x7f305c0fc7d0 msgr2=0x7f305c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 shutdown_connections 2024-09-10T10:01:53.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:53.932+0000 7f30622c8640 1 -- 172.21.15.45:0/511365612 wait complete. 2024-09-10T10:01:54.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:54 smithi045 ceph-mon[31114]: pgmap v285: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 5 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:54.570 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:54 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/511365612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:54.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:01:54.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:54 smithi162 ceph-mon[34963]: pgmap v285: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 5 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:54.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:54 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/511365612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:54.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:54 smithi047 ceph-mon[35244]: pgmap v285: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.9 KiB/s wr, 5 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:54.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:54 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/511365612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:54.854 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:55.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 -- 172.21.15.45:0/4105487454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102790 msgr2=0x7fb2101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 --2- 172.21.15.45:0/4105487454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102790 0x7fb2101067e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb200009900 tx=0x7fb20002a410 comp rx=0 tx=0).stop 2024-09-10T10:01:55.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 -- 172.21.15.45:0/4105487454 shutdown_connections 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 --2- 172.21.15.45:0/4105487454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb210106f10 0x7fb21010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 --2- 172.21.15.45:0/4105487454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210102790 0x7fb2101067e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 --2- 172.21.15.45:0/4105487454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb210101de0 0x7fb2101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 -- 172.21.15.45:0/4105487454 >> 172.21.15.45:0/4105487454 conn(0x7fb2100fd8c0 msgr2=0x7fb2100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 -- 172.21.15.45:0/4105487454 shutdown_connections 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.362+0000 7fb217b8c640 1 -- 172.21.15.45:0/4105487454 wait complete. 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 Processor -- start 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 -- start start 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb210101de0 0x7fb210198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb210102790 0x7fb210198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210106f10 0x7fb21019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb210111f70 con 0x7fb210106f10 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb210111df0 con 0x7fb210101de0 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb217b8c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101120f0 con 0x7fb210102790 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb216129640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210106f10 0x7fb21019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb216129640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210106f10 0x7fb21019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34820/0 (socket says 172.21.15.45:34820) 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb216129640 1 -- 172.21.15.45:0/2486999530 learned_addr learned my addr 172.21.15.45:0/2486999530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb215928640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb210101de0 0x7fb210198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.363+0000 7fb215127640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb210102790 0x7fb210198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb216129640 1 -- 172.21.15.45:0/2486999530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb210102790 msgr2=0x7fb210198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb216129640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb210102790 0x7fb210198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb216129640 1 -- 172.21.15.45:0/2486999530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb210101de0 msgr2=0x7fb210198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb216129640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb210101de0 0x7fb210198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb216129640 1 -- 172.21.15.45:0/2486999530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21019d3e0 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb216129640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210106f10 0x7fb21019cd00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb20800cad0 tx=0x7fb20800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb206ffd640 1 -- 172.21.15.45:0/2486999530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20800ae20 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb206ffd640 1 -- 172.21.15.45:0/2486999530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb208005020 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21019d6d0 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb206ffd640 1 -- 172.21.15.45:0/2486999530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20800f6f0 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.364+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2101a4fb0 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.365+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2101021c0 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.366+0000 7fb206ffd640 1 -- 172.21.15.45:0/2486999530 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb2080051d0 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.367+0000 7fb206ffd640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb1fc0a2c10 0x7fb1fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.367+0000 7fb215928640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb1fc0a2c10 0x7fb1fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.367+0000 7fb206ffd640 1 -- 172.21.15.45:0/2486999530 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb2080cf500 con 0x7fb210106f10 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.367+0000 7fb215928640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb1fc0a2c10 0x7fb1fc0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005d00 tx=0x7fb20c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:55.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.371+0000 7fb206ffd640 1 -- 172.21.15.45:0/2486999530 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb208014030 con 0x7fb210106f10 2024-09-10T10:01:55.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.547+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb210109b60 con 0x7fb210106f10 2024-09-10T10:01:55.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.548+0000 7fb206ffd640 1 -- 172.21.15.45:0/2486999530 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22675 (secure 0 0 0) 0x7fb20809c2b0 con 0x7fb210106f10 2024-09-10T10:01:55.551 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:55.551 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:01:50.297428+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:01:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb1fc0a2c10 msgr2=0x7fb1fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb1fc0a2c10 0x7fb1fc0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005d00 tx=0x7fb20c00b410 comp rx=0 tx=0).stop 2024-09-10T10:01:55.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210106f10 msgr2=0x7fb21019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210106f10 0x7fb21019cd00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb20800cad0 tx=0x7fb20800cf90 comp rx=0 tx=0).stop 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 shutdown_connections 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb1fc0a2c10 0x7fb1fc0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb210106f10 0x7fb21019cd00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb210102790 0x7fb210198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 --2- 172.21.15.45:0/2486999530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb210101de0 0x7fb210198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 >> 172.21.15.45:0/2486999530 conn(0x7fb2100fd8c0 msgr2=0x7fb2100fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 shutdown_connections 2024-09-10T10:01:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:55.551+0000 7fb217b8c640 1 -- 172.21.15.45:0/2486999530 wait complete. 2024-09-10T10:01:56.144 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T10:01:56.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:01:56.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:56 smithi045 ceph-mon[31114]: pgmap v286: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.0 KiB/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:56.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:56 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2486999530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:56.405 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:56.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:56 smithi162 ceph-mon[34963]: pgmap v286: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.0 KiB/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:56.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:56 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2486999530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:56 smithi047 ceph-mon[35244]: pgmap v286: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 6.0 KiB/s wr, 6 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:56.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:56 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2486999530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:01:56.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 -- 172.21.15.45:0/2800099738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4103660 msgr2=0x7f37c410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/2800099738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4103660 0x7f37c410bb50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009900 tx=0x7f37b802a480 comp rx=0 tx=0).stop 2024-09-10T10:01:56.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 -- 172.21.15.45:0/2800099738 shutdown_connections 2024-09-10T10:01:56.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/2800099738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37c410c170 0x7f37c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/2800099738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4103660 0x7f37c410bb50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/2800099738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c4102d40 0x7f37c4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 -- 172.21.15.45:0/2800099738 >> 172.21.15.45:0/2800099738 conn(0x7f37c40fc7a0 msgr2=0x7f37c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 -- 172.21.15.45:0/2800099738 shutdown_connections 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.903+0000 7f37c8ff1640 1 -- 172.21.15.45:0/2800099738 wait complete. 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 Processor -- start 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 -- start start 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4102d40 0x7f37c419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37c4103660 0x7f37c419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c410c170 0x7f37c419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111fb0 con 0x7f37c4102d40 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111e30 con 0x7f37c4103660 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c8ff1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4112130 con 0x7f37c410c170 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4102d40 0x7f37c419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4102d40 0x7f37c419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34840/0 (socket says 172.21.15.45:34840) 2024-09-10T10:01:56.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c259c640 1 -- 172.21.15.45:0/1935506129 learned_addr learned my addr 172.21.15.45:0/1935506129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.904+0000 7f37c1d9b640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37c4103660 0x7f37c419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c2d9d640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c410c170 0x7f37c419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c259c640 1 -- 172.21.15.45:0/1935506129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c410c170 msgr2=0x7f37c419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c259c640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c410c170 0x7f37c419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c259c640 1 -- 172.21.15.45:0/1935506129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37c4103660 msgr2=0x7f37c419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c259c640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37c4103660 0x7f37c419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c259c640 1 -- 172.21.15.45:0/1935506129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c419f6a0 con 0x7f37c4102d40 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c259c640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4102d40 0x7f37c419a6f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ba50 tx=0x7f37b400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37b37fe640 1 -- 172.21.15.45:0/1935506129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4019070 con 0x7f37c4102d40 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37b37fe640 1 -- 172.21.15.45:0/1935506129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b400cbe0 con 0x7f37c4102d40 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c419f990 con 0x7f37c4102d40 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37b37fe640 1 -- 172.21.15.45:0/1935506129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40076e0 con 0x7f37c4102d40 2024-09-10T10:01:56.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.905+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c41a7270 con 0x7f37c4102d40 2024-09-10T10:01:56.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.906+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c4103120 con 0x7f37c4102d40 2024-09-10T10:01:56.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.907+0000 7f37b37fe640 1 -- 172.21.15.45:0/1935506129 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f37b401c070 con 0x7f37c4102d40 2024-09-10T10:01:56.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.908+0000 7f37b37fe640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f37ac0a2c10 0x7f37ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:56.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.908+0000 7f37b37fe640 1 -- 172.21.15.45:0/1935506129 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f37b40ce6b0 con 0x7f37c4102d40 2024-09-10T10:01:56.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.908+0000 7f37c1d9b640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f37ac0a2c10 0x7f37ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:56.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.909+0000 7f37c1d9b640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f37ac0a2c10 0x7f37ac0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000c00 tx=0x7f37b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:56.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:56.912+0000 7f37b37fe640 1 -- 172.21.15.45:0/1935506129 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37b409af80 con 0x7f37c4102d40 2024-09-10T10:01:57.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.087+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37c4102060 con 0x7f37ac0a2c10 2024-09-10T10:01:57.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.094+0000 7f37b37fe640 1 -- 172.21.15.45:0/1935506129 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258638 (secure 0 0 0) 0x7f37c4102060 con 0x7f37ac0a2c10 2024-09-10T10:01:57.098 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:57.100 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-10T10:01:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f37ac0a2c10 msgr2=0x7f37ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f37ac0a2c10 0x7f37ac0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f37b8000c00 tx=0x7f37b80058d0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4102d40 msgr2=0x7f37c419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4102d40 0x7f37c419a6f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ba50 tx=0x7f37b400bf10 comp rx=0 tx=0).stop 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 shutdown_connections 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f37ac0a2c10 0x7f37ac0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37c410c170 0x7f37c419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37c4103660 0x7f37c419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 --2- 172.21.15.45:0/1935506129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37c4102d40 0x7f37c419a6f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 >> 172.21.15.45:0/1935506129 conn(0x7f37c40fc7a0 msgr2=0x7f37c4101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 shutdown_connections 2024-09-10T10:01:57.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:57.097+0000 7f37c8ff1640 1 -- 172.21.15.45:0/1935506129 wait complete. 2024-09-10T10:01:57.749 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-10T10:01:56.618865+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632465,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":168,"num_read_kb":177,"num_write":249,"num_write_kb":3000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":288,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357584,"kb_used_data":12432,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584240,"statfs":{"total":1151940427776,"available":1151574261760,"internally_reserved":0,"allocated":12730368,"data_stored":7150707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33085,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":38,"num_write":18,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001943"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335498+0000","last_change":"2024-09-10T10:01:34.253963+0000","last_active":"2024-09-10T10:01:50.335498+0000","last_peered":"2024-09-10T10:01:50.335498+0000","last_clean":"2024-09-10T10:01:50.335498+0000","last_became_active":"2024-09-10T10:01:34.252547+0000","last_became_peered":"2024-09-10T10:01:34.252547+0000","last_unstale":"2024-09-10T10:01:50.335498+0000","last_undegraded":"2024-09-10T10:01:50.335498+0000","last_fullsized":"2024-09-10T10:01:50.335498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:55.761035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321219+0000","last_change":"2024-09-10T10:01:34.249329+0000","last_active":"2024-09-10T10:01:50.321219+0000","last_peered":"2024-09-10T10:01:50.321219+0000","last_clean":"2024-09-10T10:01:50.321219+0000","last_became_active":"2024-09-10T10:01:34.249036+0000","last_became_peered":"2024-09-10T10:01:34.249036+0000","last_unstale":"2024-09-10T10:01:50.321219+0000","last_undegraded":"2024-09-10T10:01:50.321219+0000","last_fullsized":"2024-09-10T10:01:50.321219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:26.244307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320430+0000","last_change":"2024-09-10T10:01:34.242770+0000","last_active":"2024-09-10T10:01:50.320430+0000","last_peered":"2024-09-10T10:01:50.320430+0000","last_clean":"2024-09-10T10:01:50.320430+0000","last_became_active":"2024-09-10T10:01:34.242465+0000","last_became_peered":"2024-09-10T10:01:34.242465+0000","last_unstale":"2024-09-10T10:01:50.320430+0000","last_undegraded":"2024-09-10T10:01:50.320430+0000","last_fullsized":"2024-09-10T10:01:50.320430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:32.776914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838909+0000","last_change":"2024-09-10T10:01:34.249110+0000","last_active":"2024-09-10T10:01:50.838909+0000","last_peered":"2024-09-10T10:01:50.838909+0000","last_clean":"2024-09-10T10:01:50.838909+0000","last_became_active":"2024-09-10T10:01:34.248758+0000","last_became_peered":"2024-09-10T10:01:34.248758+0000","last_unstale":"2024-09-10T10:01:50.838909+0000","last_undegraded":"2024-09-10T10:01:50.838909+0000","last_fullsized":"2024-09-10T10:01:50.838909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:51.622686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338095+0000","last_change":"2024-09-10T10:01:34.248799+0000","last_active":"2024-09-10T10:01:50.338095+0000","last_peered":"2024-09-10T10:01:50.338095+0000","last_clean":"2024-09-10T10:01:50.338095+0000","last_became_active":"2024-09-10T10:01:34.246797+0000","last_became_peered":"2024-09-10T10:01:34.246797+0000","last_unstale":"2024-09-10T10:01:50.338095+0000","last_undegraded":"2024-09-10T10:01:50.338095+0000","last_fullsized":"2024-09-10T10:01:50.338095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:33.887465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320541+0000","last_change":"2024-09-10T10:01:34.252139+0000","last_active":"2024-09-10T10:01:50.320541+0000","last_peered":"2024-09-10T10:01:50.320541+0000","last_clean":"2024-09-10T10:01:50.320541+0000","last_became_active":"2024-09-10T10:01:34.251823+0000","last_became_peered":"2024-09-10T10:01:34.251823+0000","last_unstale":"2024-09-10T10:01:50.320541+0000","last_undegraded":"2024-09-10T10:01:50.320541+0000","last_fullsized":"2024-09-10T10:01:50.320541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:31.642428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337590+0000","last_change":"2024-09-10T10:01:34.258097+0000","last_active":"2024-09-10T10:01:50.337590+0000","last_peered":"2024-09-10T10:01:50.337590+0000","last_clean":"2024-09-10T10:01:50.337590+0000","last_became_active":"2024-09-10T10:01:34.257717+0000","last_became_peered":"2024-09-10T10:01:34.257717+0000","last_unstale":"2024-09-10T10:01:50.337590+0000","last_undegraded":"2024-09-10T10:01:50.337590+0000","last_fullsized":"2024-09-10T10:01:50.337590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:16.280503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319894+0000","last_change":"2024-09-10T10:01:34.249901+0000","last_active":"2024-09-10T10:01:50.319894+0000","last_peered":"2024-09-10T10:01:50.319894+0000","last_clean":"2024-09-10T10:01:50.319894+0000","last_became_active":"2024-09-10T10:01:34.249361+0000","last_became_peered":"2024-09-10T10:01:34.249361+0000","last_unstale":"2024-09-10T10:01:50.319894+0000","last_undegraded":"2024-09-10T10:01:50.319894+0000","last_fullsized":"2024-09-10T10:01:50.319894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:42.282621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.711941+0000","last_change":"2024-09-10T10:01:50.711491+0000","last_active":"2024-09-10T10:01:50.711941+0000","last_peered":"2024-09-10T10:01:50.711941+0000","last_clean":"2024-09-10T10:01:50.711941+0000","last_became_active":"2024-09-10T10:01:50.711162+0000","last_became_peered":"2024-09-10T10:01:50.711162+0000","last_unstale":"2024-09-10T10:01:50.711941+0000","last_undegraded":"2024-09-10T10:01:50.711941+0000","last_fullsized":"2024-09-10T10:01:50.711941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:08.502816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320934+0000","last_change":"2024-09-10T10:01:34.252538+0000","last_active":"2024-09-10T10:01:50.320934+0000","last_peered":"2024-09-10T10:01:50.320934+0000","last_clean":"2024-09-10T10:01:50.320934+0000","last_became_active":"2024-09-10T10:01:34.251384+0000","last_became_peered":"2024-09-10T10:01:34.251384+0000","last_unstale":"2024-09-10T10:01:50.320934+0000","last_undegraded":"2024-09-10T10:01:50.320934+0000","last_fullsized":"2024-09-10T10:01:50.320934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:39.990996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338061+0000","last_change":"2024-09-10T10:01:34.254262+0000","last_active":"2024-09-10T10:01:50.338061+0000","last_peered":"2024-09-10T10:01:50.338061+0000","last_clean":"2024-09-10T10:01:50.338061+0000","last_became_active":"2024-09-10T10:01:34.253907+0000","last_became_peered":"2024-09-10T10:01:34.253907+0000","last_unstale":"2024-09-10T10:01:50.338061+0000","last_undegraded":"2024-09-10T10:01:50.338061+0000","last_fullsized":"2024-09-10T10:01:50.338061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:38.681367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838921+0000","last_change":"2024-09-10T10:01:34.252194+0000","last_active":"2024-09-10T10:01:50.838921+0000","last_peered":"2024-09-10T10:01:50.838921+0000","last_clean":"2024-09-10T10:01:50.838921+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.838921+0000","last_undegraded":"2024-09-10T10:01:50.838921+0000","last_fullsized":"2024-09-10T10:01:50.838921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:47.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332280+0000","last_change":"2024-09-10T10:01:34.251795+0000","last_active":"2024-09-10T10:01:50.332280+0000","last_peered":"2024-09-10T10:01:50.332280+0000","last_clean":"2024-09-10T10:01:50.332280+0000","last_became_active":"2024-09-10T10:01:34.251403+0000","last_became_peered":"2024-09-10T10:01:34.251403+0000","last_unstale":"2024-09-10T10:01:50.332280+0000","last_undegraded":"2024-09-10T10:01:50.332280+0000","last_fullsized":"2024-09-10T10:01:50.332280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:11.650068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321079+0000","last_change":"2024-09-10T10:01:34.252718+0000","last_active":"2024-09-10T10:01:50.321079+0000","last_peered":"2024-09-10T10:01:50.321079+0000","last_clean":"2024-09-10T10:01:50.321079+0000","last_became_active":"2024-09-10T10:01:34.252415+0000","last_became_peered":"2024-09-10T10:01:34.252415+0000","last_unstale":"2024-09-10T10:01:50.321079+0000","last_undegraded":"2024-09-10T10:01:50.321079+0000","last_fullsized":"2024-09-10T10:01:50.321079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:51.057191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320828+0000","last_change":"2024-09-10T10:01:34.252237+0000","last_active":"2024-09-10T10:01:50.320828+0000","last_peered":"2024-09-10T10:01:50.320828+0000","last_clean":"2024-09-10T10:01:50.320828+0000","last_became_active":"2024-09-10T10:01:34.251909+0000","last_became_peered":"2024-09-10T10:01:34.251909+0000","last_unstale":"2024-09-10T10:01:50.320828+0000","last_undegraded":"2024-09-10T10:01:50.320828+0000","last_fullsized":"2024-09-10T10:01:50.320828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:15.150503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318205+0000","last_change":"2024-09-10T10:01:34.259107+0000","last_active":"2024-09-10T10:01:50.318205+0000","last_peered":"2024-09-10T10:01:50.318205+0000","last_clean":"2024-09-10T10:01:50.318205+0000","last_became_active":"2024-09-10T10:01:34.258230+0000","last_became_peered":"2024-09-10T10:01:34.258230+0000","last_unstale":"2024-09-10T10:01:50.318205+0000","last_undegraded":"2024-09-10T10:01:50.318205+0000","last_fullsized":"2024-09-10T10:01:50.318205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:36.091358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335573+0000","last_change":"2024-09-10T10:01:34.254341+0000","last_active":"2024-09-10T10:01:50.335573+0000","last_peered":"2024-09-10T10:01:50.335573+0000","last_clean":"2024-09-10T10:01:50.335573+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.335573+0000","last_undegraded":"2024-09-10T10:01:50.335573+0000","last_fullsized":"2024-09-10T10:01:50.335573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:46.668715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317729+0000","last_change":"2024-09-10T10:01:34.247950+0000","last_active":"2024-09-10T10:01:50.317729+0000","last_peered":"2024-09-10T10:01:50.317729+0000","last_clean":"2024-09-10T10:01:50.317729+0000","last_became_active":"2024-09-10T10:01:34.247659+0000","last_became_peered":"2024-09-10T10:01:34.247659+0000","last_unstale":"2024-09-10T10:01:50.317729+0000","last_undegraded":"2024-09-10T10:01:50.317729+0000","last_fullsized":"2024-09-10T10:01:50.317729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:34.113605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333284+0000","last_change":"2024-09-10T10:01:34.248153+0000","last_active":"2024-09-10T10:01:50.333284+0000","last_peered":"2024-09-10T10:01:50.333284+0000","last_clean":"2024-09-10T10:01:50.333284+0000","last_became_active":"2024-09-10T10:01:34.247773+0000","last_became_peered":"2024-09-10T10:01:34.247773+0000","last_unstale":"2024-09-10T10:01:50.333284+0000","last_undegraded":"2024-09-10T10:01:50.333284+0000","last_fullsized":"2024-09-10T10:01:50.333284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:54.223519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335273+0000","last_change":"2024-09-10T10:01:34.252781+0000","last_active":"2024-09-10T10:01:50.335273+0000","last_peered":"2024-09-10T10:01:50.335273+0000","last_clean":"2024-09-10T10:01:50.335273+0000","last_became_active":"2024-09-10T10:01:34.252459+0000","last_became_peered":"2024-09-10T10:01:34.252459+0000","last_unstale":"2024-09-10T10:01:50.335273+0000","last_undegraded":"2024-09-10T10:01:50.335273+0000","last_fullsized":"2024-09-10T10:01:50.335273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:32.039769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332286+0000","last_change":"2024-09-10T10:01:34.251147+0000","last_active":"2024-09-10T10:01:50.332286+0000","last_peered":"2024-09-10T10:01:50.332286+0000","last_clean":"2024-09-10T10:01:50.332286+0000","last_became_active":"2024-09-10T10:01:34.250835+0000","last_became_peered":"2024-09-10T10:01:34.250835+0000","last_unstale":"2024-09-10T10:01:50.332286+0000","last_undegraded":"2024-09-10T10:01:50.332286+0000","last_fullsized":"2024-09-10T10:01:50.332286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:56.914798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318417+0000","last_change":"2024-09-10T10:01:34.259300+0000","last_active":"2024-09-10T10:01:50.318417+0000","last_peered":"2024-09-10T10:01:50.318417+0000","last_clean":"2024-09-10T10:01:50.318417+0000","last_became_active":"2024-09-10T10:01:34.258192+0000","last_became_peered":"2024-09-10T10:01:34.258192+0000","last_unstale":"2024-09-10T10:01:50.318417+0000","last_undegraded":"2024-09-10T10:01:50.318417+0000","last_fullsized":"2024-09-10T10:01:50.318417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:19.755696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319962+0000","last_change":"2024-09-10T10:01:34.244797+0000","last_active":"2024-09-10T10:01:50.319962+0000","last_peered":"2024-09-10T10:01:50.319962+0000","last_clean":"2024-09-10T10:01:50.319962+0000","last_became_active":"2024-09-10T10:01:34.244464+0000","last_became_peered":"2024-09-10T10:01:34.244464+0000","last_unstale":"2024-09-10T10:01:50.319962+0000","last_undegraded":"2024-09-10T10:01:50.319962+0000","last_fullsized":"2024-09-10T10:01:50.319962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:23.750063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710213+0000","last_change":"2024-09-10T10:01:34.248445+0000","last_active":"2024-09-10T10:01:50.710213+0000","last_peered":"2024-09-10T10:01:50.710213+0000","last_clean":"2024-09-10T10:01:50.710213+0000","last_became_active":"2024-09-10T10:01:34.248020+0000","last_became_peered":"2024-09-10T10:01:34.248020+0000","last_unstale":"2024-09-10T10:01:50.710213+0000","last_undegraded":"2024-09-10T10:01:50.710213+0000","last_fullsized":"2024-09-10T10:01:50.710213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:32.096061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339780+0000","last_change":"2024-09-10T10:01:50.339780+0000","last_active":"2024-09-10T10:01:50.339780+0000","last_peered":"2024-09-10T10:01:50.339780+0000","last_clean":"2024-09-10T10:01:50.339780+0000","last_became_active":"2024-09-10T10:01:50.339519+0000","last_became_peered":"2024-09-10T10:01:50.339519+0000","last_unstale":"2024-09-10T10:01:50.339780+0000","last_undegraded":"2024-09-10T10:01:50.339780+0000","last_fullsized":"2024-09-10T10:01:50.339780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:27.368296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333351+0000","last_change":"2024-09-10T10:01:34.244512+0000","last_active":"2024-09-10T10:01:50.333351+0000","last_peered":"2024-09-10T10:01:50.333351+0000","last_clean":"2024-09-10T10:01:50.333351+0000","last_became_active":"2024-09-10T10:01:34.244191+0000","last_became_peered":"2024-09-10T10:01:34.244191+0000","last_unstale":"2024-09-10T10:01:50.333351+0000","last_undegraded":"2024-09-10T10:01:50.333351+0000","last_fullsized":"2024-09-10T10:01:50.333351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:19.019159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335673+0000","last_change":"2024-09-10T10:01:34.248683+0000","last_active":"2024-09-10T10:01:50.335673+0000","last_peered":"2024-09-10T10:01:50.335673+0000","last_clean":"2024-09-10T10:01:50.335673+0000","last_became_active":"2024-09-10T10:01:34.248228+0000","last_became_peered":"2024-09-10T10:01:34.248228+0000","last_unstale":"2024-09-10T10:01:50.335673+0000","last_undegraded":"2024-09-10T10:01:50.335673+0000","last_fullsized":"2024-09-10T10:01:50.335673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:00.485621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332452+0000","last_change":"2024-09-10T10:01:34.254683+0000","last_active":"2024-09-10T10:01:50.332452+0000","last_peered":"2024-09-10T10:01:50.332452+0000","last_clean":"2024-09-10T10:01:50.332452+0000","last_became_active":"2024-09-10T10:01:34.254384+0000","last_became_peered":"2024-09-10T10:01:34.254384+0000","last_unstale":"2024-09-10T10:01:50.332452+0000","last_undegraded":"2024-09-10T10:01:50.332452+0000","last_fullsized":"2024-09-10T10:01:50.332452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:35.404584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321448+0000","last_change":"2024-09-10T10:01:34.254079+0000","last_active":"2024-09-10T10:01:50.321448+0000","last_peered":"2024-09-10T10:01:50.321448+0000","last_clean":"2024-09-10T10:01:50.321448+0000","last_became_active":"2024-09-10T10:01:34.253527+0000","last_became_peered":"2024-09-10T10:01:34.253527+0000","last_unstale":"2024-09-10T10:01:50.321448+0000","last_undegraded":"2024-09-10T10:01:50.321448+0000","last_fullsized":"2024-09-10T10:01:50.321448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:31.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332948+0000","last_change":"2024-09-10T10:01:34.244840+0000","last_active":"2024-09-10T10:01:50.332948+0000","last_peered":"2024-09-10T10:01:50.332948+0000","last_clean":"2024-09-10T10:01:50.332948+0000","last_became_active":"2024-09-10T10:01:34.244482+0000","last_became_peered":"2024-09-10T10:01:34.244482+0000","last_unstale":"2024-09-10T10:01:50.332948+0000","last_undegraded":"2024-09-10T10:01:50.332948+0000","last_fullsized":"2024-09-10T10:01:50.332948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:50.666840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332939+0000","last_change":"2024-09-10T10:01:34.250789+0000","last_active":"2024-09-10T10:01:50.332939+0000","last_peered":"2024-09-10T10:01:50.332939+0000","last_clean":"2024-09-10T10:01:50.332939+0000","last_became_active":"2024-09-10T10:01:34.250445+0000","last_became_peered":"2024-09-10T10:01:34.250445+0000","last_unstale":"2024-09-10T10:01:50.332939+0000","last_undegraded":"2024-09-10T10:01:50.332939+0000","last_fullsized":"2024-09-10T10:01:50.332939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:00.389643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335618+0000","last_change":"2024-09-10T10:01:34.253139+0000","last_active":"2024-09-10T10:01:50.335618+0000","last_peered":"2024-09-10T10:01:50.335618+0000","last_clean":"2024-09-10T10:01:50.335618+0000","last_became_active":"2024-09-10T10:01:34.251801+0000","last_became_peered":"2024-09-10T10:01:34.251801+0000","last_unstale":"2024-09-10T10:01:50.335618+0000","last_undegraded":"2024-09-10T10:01:50.335618+0000","last_fullsized":"2024-09-10T10:01:50.335618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:33.492291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338268+0000","last_change":"2024-09-10T10:01:34.243669+0000","last_active":"2024-09-10T10:01:50.338268+0000","last_peered":"2024-09-10T10:01:50.338268+0000","last_clean":"2024-09-10T10:01:50.338268+0000","last_became_active":"2024-09-10T10:01:34.243291+0000","last_became_peered":"2024-09-10T10:01:34.243291+0000","last_unstale":"2024-09-10T10:01:50.338268+0000","last_undegraded":"2024-09-10T10:01:50.338268+0000","last_fullsized":"2024-09-10T10:01:50.338268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:47.157495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710729+0000","last_change":"2024-09-10T10:01:32.206317+0000","last_active":"2024-09-10T10:01:50.710729+0000","last_peered":"2024-09-10T10:01:50.710729+0000","last_clean":"2024-09-10T10:01:50.710729+0000","last_became_active":"2024-09-10T10:01:32.205412+0000","last_became_peered":"2024-09-10T10:01:32.205412+0000","last_unstale":"2024-09-10T10:01:50.710729+0000","last_undegraded":"2024-09-10T10:01:50.710729+0000","last_fullsized":"2024-09-10T10:01:50.710729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:07.489923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710763+0000","last_change":"2024-09-10T10:01:34.254783+0000","last_active":"2024-09-10T10:01:50.710763+0000","last_peered":"2024-09-10T10:01:50.710763+0000","last_clean":"2024-09-10T10:01:50.710763+0000","last_became_active":"2024-09-10T10:01:34.254465+0000","last_became_peered":"2024-09-10T10:01:34.254465+0000","last_unstale":"2024-09-10T10:01:50.710763+0000","last_undegraded":"2024-09-10T10:01:50.710763+0000","last_fullsized":"2024-09-10T10:01:50.710763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:57.983265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320605+0000","last_change":"2024-09-10T10:01:32.204551+0000","last_active":"2024-09-10T10:01:50.320605+0000","last_peered":"2024-09-10T10:01:50.320605+0000","last_clean":"2024-09-10T10:01:50.320605+0000","last_became_active":"2024-09-10T10:01:32.204214+0000","last_became_peered":"2024-09-10T10:01:32.204214+0000","last_unstale":"2024-09-10T10:01:50.320605+0000","last_undegraded":"2024-09-10T10:01:50.320605+0000","last_fullsized":"2024-09-10T10:01:50.320605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:50.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332156+0000","last_change":"2024-09-10T10:01:32.205907+0000","last_active":"2024-09-10T10:01:50.332156+0000","last_peered":"2024-09-10T10:01:50.332156+0000","last_clean":"2024-09-10T10:01:50.332156+0000","last_became_active":"2024-09-10T10:01:32.205523+0000","last_became_peered":"2024-09-10T10:01:32.205523+0000","last_unstale":"2024-09-10T10:01:50.332156+0000","last_undegraded":"2024-09-10T10:01:50.332156+0000","last_fullsized":"2024-09-10T10:01:50.332156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:14.540234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319983+0000","last_change":"2024-09-10T10:01:34.249826+0000","last_active":"2024-09-10T10:01:50.319983+0000","last_peered":"2024-09-10T10:01:50.319983+0000","last_clean":"2024-09-10T10:01:50.319983+0000","last_became_active":"2024-09-10T10:01:34.248940+0000","last_became_peered":"2024-09-10T10:01:34.248940+0000","last_unstale":"2024-09-10T10:01:50.319983+0000","last_undegraded":"2024-09-10T10:01:50.319983+0000","last_fullsized":"2024-09-10T10:01:50.319983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:49.125909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710836+0000","last_change":"2024-09-10T10:01:32.205559+0000","last_active":"2024-09-10T10:01:50.710836+0000","last_peered":"2024-09-10T10:01:50.710836+0000","last_clean":"2024-09-10T10:01:50.710836+0000","last_became_active":"2024-09-10T10:01:32.205251+0000","last_became_peered":"2024-09-10T10:01:32.205251+0000","last_unstale":"2024-09-10T10:01:50.710836+0000","last_undegraded":"2024-09-10T10:01:50.710836+0000","last_fullsized":"2024-09-10T10:01:50.710836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:11.442234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317525+0000","last_change":"2024-09-10T10:01:34.245852+0000","last_active":"2024-09-10T10:01:50.317525+0000","last_peered":"2024-09-10T10:01:50.317525+0000","last_clean":"2024-09-10T10:01:50.317525+0000","last_became_active":"2024-09-10T10:01:34.245487+0000","last_became_peered":"2024-09-10T10:01:34.245487+0000","last_unstale":"2024-09-10T10:01:50.317525+0000","last_undegraded":"2024-09-10T10:01:50.317525+0000","last_fullsized":"2024-09-10T10:01:50.317525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:01.995443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320839+0000","last_change":"2024-09-10T10:01:32.202754+0000","last_active":"2024-09-10T10:01:50.320839+0000","last_peered":"2024-09-10T10:01:50.320839+0000","last_clean":"2024-09-10T10:01:50.320839+0000","last_became_active":"2024-09-10T10:01:32.202360+0000","last_became_peered":"2024-09-10T10:01:32.202360+0000","last_unstale":"2024-09-10T10:01:50.320839+0000","last_undegraded":"2024-09-10T10:01:50.320839+0000","last_fullsized":"2024-09-10T10:01:50.320839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:04.864276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318627+0000","last_change":"2024-09-10T10:01:34.245670+0000","last_active":"2024-09-10T10:01:50.318627+0000","last_peered":"2024-09-10T10:01:50.318627+0000","last_clean":"2024-09-10T10:01:50.318627+0000","last_became_active":"2024-09-10T10:01:34.244922+0000","last_became_peered":"2024-09-10T10:01:34.244922+0000","last_unstale":"2024-09-10T10:01:50.318627+0000","last_undegraded":"2024-09-10T10:01:50.318627+0000","last_fullsized":"2024-09-10T10:01:50.318627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:05.442144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338108+0000","last_change":"2024-09-10T10:01:32.204658+0000","last_active":"2024-09-10T10:01:50.338108+0000","last_peered":"2024-09-10T10:01:50.338108+0000","last_clean":"2024-09-10T10:01:50.338108+0000","last_became_active":"2024-09-10T10:01:32.204310+0000","last_became_peered":"2024-09-10T10:01:32.204310+0000","last_unstale":"2024-09-10T10:01:50.338108+0000","last_undegraded":"2024-09-10T10:01:50.338108+0000","last_fullsized":"2024-09-10T10:01:50.338108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:16.446977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.712239+0000","last_change":"2024-09-10T10:01:50.712239+0000","last_active":"2024-09-10T10:01:50.712239+0000","last_peered":"2024-09-10T10:01:50.712239+0000","last_clean":"2024-09-10T10:01:50.712239+0000","last_became_active":"2024-09-10T10:01:50.711826+0000","last_became_peered":"2024-09-10T10:01:50.711826+0000","last_unstale":"2024-09-10T10:01:50.712239+0000","last_undegraded":"2024-09-10T10:01:50.712239+0000","last_fullsized":"2024-09-10T10:01:50.712239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:54.670359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710657+0000","last_change":"2024-09-10T10:01:32.530649+0000","last_active":"2024-09-10T10:01:50.710657+0000","last_peered":"2024-09-10T10:01:50.710657+0000","last_clean":"2024-09-10T10:01:50.710657+0000","last_became_active":"2024-09-10T10:01:32.530157+0000","last_became_peered":"2024-09-10T10:01:32.530157+0000","last_unstale":"2024-09-10T10:01:50.710657+0000","last_undegraded":"2024-09-10T10:01:50.710657+0000","last_fullsized":"2024-09-10T10:01:50.710657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:30.235163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339461+0000","last_change":"2024-09-10T10:01:50.339461+0000","last_active":"2024-09-10T10:01:50.339461+0000","last_peered":"2024-09-10T10:01:50.339461+0000","last_clean":"2024-09-10T10:01:50.339461+0000","last_became_active":"2024-09-10T10:01:50.339158+0000","last_became_peered":"2024-09-10T10:01:50.339158+0000","last_unstale":"2024-09-10T10:01:50.339461+0000","last_undegraded":"2024-09-10T10:01:50.339461+0000","last_fullsized":"2024-09-10T10:01:50.339461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:11.733302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'19","reported_seq":67,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:51.294023+0000","last_change":"2024-09-10T10:01:32.205933+0000","last_active":"2024-09-10T10:01:51.294023+0000","last_peered":"2024-09-10T10:01:51.294023+0000","last_clean":"2024-09-10T10:01:51.294023+0000","last_became_active":"2024-09-10T10:01:32.205592+0000","last_became_peered":"2024-09-10T10:01:32.205592+0000","last_unstale":"2024-09-10T10:01:51.294023+0000","last_undegraded":"2024-09-10T10:01:51.294023+0000","last_fullsized":"2024-09-10T10:01:51.294023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:29.107388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41065,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":52,"num_write":19,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338800+0000","last_change":"2024-09-10T10:01:34.249720+0000","last_active":"2024-09-10T10:01:50.338800+0000","last_peered":"2024-09-10T10:01:50.338800+0000","last_clean":"2024-09-10T10:01:50.338800+0000","last_became_active":"2024-09-10T10:01:34.249285+0000","last_became_peered":"2024-09-10T10:01:34.249285+0000","last_unstale":"2024-09-10T10:01:50.338800+0000","last_undegraded":"2024-09-10T10:01:50.338800+0000","last_fullsized":"2024-09-10T10:01:50.338800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:30.787344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333124+0000","last_change":"2024-09-10T10:01:32.202622+0000","last_active":"2024-09-10T10:01:50.333124+0000","last_peered":"2024-09-10T10:01:50.333124+0000","last_clean":"2024-09-10T10:01:50.333124+0000","last_became_active":"2024-09-10T10:01:32.200870+0000","last_became_peered":"2024-09-10T10:01:32.200870+0000","last_unstale":"2024-09-10T10:01:50.333124+0000","last_undegraded":"2024-09-10T10:01:50.333124+0000","last_fullsized":"2024-09-10T10:01:50.333124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:48.562971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332189+0000","last_change":"2024-09-10T10:01:34.252622+0000","last_active":"2024-09-10T10:01:50.332189+0000","last_peered":"2024-09-10T10:01:50.332189+0000","last_clean":"2024-09-10T10:01:50.332189+0000","last_became_active":"2024-09-10T10:01:34.252271+0000","last_became_peered":"2024-09-10T10:01:34.252271+0000","last_unstale":"2024-09-10T10:01:50.332189+0000","last_undegraded":"2024-09-10T10:01:50.332189+0000","last_fullsized":"2024-09-10T10:01:50.332189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:03.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710592+0000","last_change":"2024-09-10T10:01:32.206133+0000","last_active":"2024-09-10T10:01:50.710592+0000","last_peered":"2024-09-10T10:01:50.710592+0000","last_clean":"2024-09-10T10:01:50.710592+0000","last_became_active":"2024-09-10T10:01:32.205747+0000","last_became_peered":"2024-09-10T10:01:32.205747+0000","last_unstale":"2024-09-10T10:01:50.710592+0000","last_undegraded":"2024-09-10T10:01:50.710592+0000","last_fullsized":"2024-09-10T10:01:50.710592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:30.515614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337830+0000","last_change":"2024-09-10T10:01:34.257702+0000","last_active":"2024-09-10T10:01:50.337830+0000","last_peered":"2024-09-10T10:01:50.337830+0000","last_clean":"2024-09-10T10:01:50.337830+0000","last_became_active":"2024-09-10T10:01:34.257366+0000","last_became_peered":"2024-09-10T10:01:34.257366+0000","last_unstale":"2024-09-10T10:01:50.337830+0000","last_undegraded":"2024-09-10T10:01:50.337830+0000","last_fullsized":"2024-09-10T10:01:50.337830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:51.772522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838725+0000","last_change":"2024-09-10T10:01:32.205527+0000","last_active":"2024-09-10T10:01:50.838725+0000","last_peered":"2024-09-10T10:01:50.838725+0000","last_clean":"2024-09-10T10:01:50.838725+0000","last_became_active":"2024-09-10T10:01:32.205162+0000","last_became_peered":"2024-09-10T10:01:32.205162+0000","last_unstale":"2024-09-10T10:01:50.838725+0000","last_undegraded":"2024-09-10T10:01:50.838725+0000","last_fullsized":"2024-09-10T10:01:50.838725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:28.149331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337435+0000","last_change":"2024-09-10T10:01:34.250691+0000","last_active":"2024-09-10T10:01:50.337435+0000","last_peered":"2024-09-10T10:01:50.337435+0000","last_clean":"2024-09-10T10:01:50.337435+0000","last_became_active":"2024-09-10T10:01:34.249303+0000","last_became_peered":"2024-09-10T10:01:34.249303+0000","last_unstale":"2024-09-10T10:01:50.337435+0000","last_undegraded":"2024-09-10T10:01:50.337435+0000","last_fullsized":"2024-09-10T10:01:50.337435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:47.413567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338071+0000","last_change":"2024-09-10T10:01:32.530043+0000","last_active":"2024-09-10T10:01:50.338071+0000","last_peered":"2024-09-10T10:01:50.338071+0000","last_clean":"2024-09-10T10:01:50.338071+0000","last_became_active":"2024-09-10T10:01:32.529502+0000","last_became_peered":"2024-09-10T10:01:32.529502+0000","last_unstale":"2024-09-10T10:01:50.338071+0000","last_undegraded":"2024-09-10T10:01:50.338071+0000","last_fullsized":"2024-09-10T10:01:50.338071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:26.133771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710304+0000","last_change":"2024-09-10T10:01:34.254722+0000","last_active":"2024-09-10T10:01:50.710304+0000","last_peered":"2024-09-10T10:01:50.710304+0000","last_clean":"2024-09-10T10:01:50.710304+0000","last_became_active":"2024-09-10T10:01:34.254390+0000","last_became_peered":"2024-09-10T10:01:34.254390+0000","last_unstale":"2024-09-10T10:01:50.710304+0000","last_undegraded":"2024-09-10T10:01:50.710304+0000","last_fullsized":"2024-09-10T10:01:50.710304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:37.261256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339061+0000","last_change":"2024-09-10T10:01:32.200744+0000","last_active":"2024-09-10T10:01:50.339061+0000","last_peered":"2024-09-10T10:01:50.339061+0000","last_clean":"2024-09-10T10:01:50.339061+0000","last_became_active":"2024-09-10T10:01:32.199906+0000","last_became_peered":"2024-09-10T10:01:32.199906+0000","last_unstale":"2024-09-10T10:01:50.339061+0000","last_undegraded":"2024-09-10T10:01:50.339061+0000","last_fullsized":"2024-09-10T10:01:50.339061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:17.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.840135+0000","last_change":"2024-09-10T10:01:50.840135+0000","last_active":"2024-09-10T10:01:50.840135+0000","last_peered":"2024-09-10T10:01:50.840135+0000","last_clean":"2024-09-10T10:01:50.840135+0000","last_became_active":"2024-09-10T10:01:50.839693+0000","last_became_peered":"2024-09-10T10:01:50.839693+0000","last_unstale":"2024-09-10T10:01:50.840135+0000","last_undegraded":"2024-09-10T10:01:50.840135+0000","last_fullsized":"2024-09-10T10:01:50.840135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:01.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332542+0000","last_change":"2024-09-10T10:01:32.200383+0000","last_active":"2024-09-10T10:01:50.332542+0000","last_peered":"2024-09-10T10:01:50.332542+0000","last_clean":"2024-09-10T10:01:50.332542+0000","last_became_active":"2024-09-10T10:01:32.199926+0000","last_became_peered":"2024-09-10T10:01:32.199926+0000","last_unstale":"2024-09-10T10:01:50.332542+0000","last_undegraded":"2024-09-10T10:01:50.332542+0000","last_fullsized":"2024-09-10T10:01:50.332542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:29.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338221+0000","last_change":"2024-09-10T10:01:34.250077+0000","last_active":"2024-09-10T10:01:50.338221+0000","last_peered":"2024-09-10T10:01:50.338221+0000","last_clean":"2024-09-10T10:01:50.338221+0000","last_became_active":"2024-09-10T10:01:34.249697+0000","last_became_peered":"2024-09-10T10:01:34.249697+0000","last_unstale":"2024-09-10T10:01:50.338221+0000","last_undegraded":"2024-09-10T10:01:50.338221+0000","last_fullsized":"2024-09-10T10:01:50.338221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:25.068115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710730+0000","last_change":"2024-09-10T10:01:32.200235+0000","last_active":"2024-09-10T10:01:50.710730+0000","last_peered":"2024-09-10T10:01:50.710730+0000","last_clean":"2024-09-10T10:01:50.710730+0000","last_became_active":"2024-09-10T10:01:32.199920+0000","last_became_peered":"2024-09-10T10:01:32.199920+0000","last_unstale":"2024-09-10T10:01:50.710730+0000","last_undegraded":"2024-09-10T10:01:50.710730+0000","last_fullsized":"2024-09-10T10:01:50.710730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:10.980054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338054+0000","last_change":"2024-09-10T10:01:34.256937+0000","last_active":"2024-09-10T10:01:50.338054+0000","last_peered":"2024-09-10T10:01:50.338054+0000","last_clean":"2024-09-10T10:01:50.338054+0000","last_became_active":"2024-09-10T10:01:34.256376+0000","last_became_peered":"2024-09-10T10:01:34.256376+0000","last_unstale":"2024-09-10T10:01:50.338054+0000","last_undegraded":"2024-09-10T10:01:50.338054+0000","last_fullsized":"2024-09-10T10:01:50.338054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:07.386683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333436+0000","last_change":"2024-09-10T10:01:32.530794+0000","last_active":"2024-09-10T10:01:50.333436+0000","last_peered":"2024-09-10T10:01:50.333436+0000","last_clean":"2024-09-10T10:01:50.333436+0000","last_became_active":"2024-09-10T10:01:32.530335+0000","last_became_peered":"2024-09-10T10:01:32.530335+0000","last_unstale":"2024-09-10T10:01:50.333436+0000","last_undegraded":"2024-09-10T10:01:50.333436+0000","last_fullsized":"2024-09-10T10:01:50.333436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:40.883029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338123+0000","last_change":"2024-09-10T10:01:34.250228+0000","last_active":"2024-09-10T10:01:50.338123+0000","last_peered":"2024-09-10T10:01:50.338123+0000","last_clean":"2024-09-10T10:01:50.338123+0000","last_became_active":"2024-09-10T10:01:34.249901+0000","last_became_peered":"2024-09-10T10:01:34.249901+0000","last_unstale":"2024-09-10T10:01:50.338123+0000","last_undegraded":"2024-09-10T10:01:50.338123+0000","last_fullsized":"2024-09-10T10:01:50.338123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:10.710185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337584+0000","last_change":"2024-09-10T10:00:23.346149+0000","last_active":"2024-09-10T10:01:50.337584+0000","last_peered":"2024-09-10T10:01:50.337584+0000","last_clean":"2024-09-10T10:01:50.337584+0000","last_became_active":"2024-09-10T10:00:23.345608+0000","last_became_peered":"2024-09-10T10:00:23.345608+0000","last_unstale":"2024-09-10T10:01:50.337584+0000","last_undegraded":"2024-09-10T10:01:50.337584+0000","last_fullsized":"2024-09-10T10:01:50.337584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_clean_scrub_stamp":"2024-09-10T09:57:03.194756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:14.092838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838755+0000","last_change":"2024-09-10T10:01:32.200726+0000","last_active":"2024-09-10T10:01:50.838755+0000","last_peered":"2024-09-10T10:01:50.838755+0000","last_clean":"2024-09-10T10:01:50.838755+0000","last_became_active":"2024-09-10T10:01:32.200163+0000","last_became_peered":"2024-09-10T10:01:32.200163+0000","last_unstale":"2024-09-10T10:01:50.838755+0000","last_undegraded":"2024-09-10T10:01:50.838755+0000","last_fullsized":"2024-09-10T10:01:50.838755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:05.300476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320505+0000","last_change":"2024-09-10T10:01:34.243258+0000","last_active":"2024-09-10T10:01:50.320505+0000","last_peered":"2024-09-10T10:01:50.320505+0000","last_clean":"2024-09-10T10:01:50.320505+0000","last_became_active":"2024-09-10T10:01:34.242060+0000","last_became_peered":"2024-09-10T10:01:34.242060+0000","last_unstale":"2024-09-10T10:01:50.320505+0000","last_undegraded":"2024-09-10T10:01:50.320505+0000","last_fullsized":"2024-09-10T10:01:50.320505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:02.910538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337582+0000","last_change":"2024-09-10T10:01:32.205174+0000","last_active":"2024-09-10T10:01:50.337582+0000","last_peered":"2024-09-10T10:01:50.337582+0000","last_clean":"2024-09-10T10:01:50.337582+0000","last_became_active":"2024-09-10T10:01:32.204121+0000","last_became_peered":"2024-09-10T10:01:32.204121+0000","last_unstale":"2024-09-10T10:01:50.337582+0000","last_undegraded":"2024-09-10T10:01:50.337582+0000","last_fullsized":"2024-09-10T10:01:50.337582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:42.533591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332204+0000","last_change":"2024-09-10T10:01:34.254895+0000","last_active":"2024-09-10T10:01:50.332204+0000","last_peered":"2024-09-10T10:01:50.332204+0000","last_clean":"2024-09-10T10:01:50.332204+0000","last_became_active":"2024-09-10T10:01:34.254572+0000","last_became_peered":"2024-09-10T10:01:34.254572+0000","last_unstale":"2024-09-10T10:01:50.332204+0000","last_undegraded":"2024-09-10T10:01:50.332204+0000","last_fullsized":"2024-09-10T10:01:50.332204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:44.194941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332304+0000","last_change":"2024-09-10T10:01:32.530093+0000","last_active":"2024-09-10T10:01:50.332304+0000","last_peered":"2024-09-10T10:01:50.332304+0000","last_clean":"2024-09-10T10:01:50.332304+0000","last_became_active":"2024-09-10T10:01:32.529756+0000","last_became_peered":"2024-09-10T10:01:32.529756+0000","last_unstale":"2024-09-10T10:01:50.332304+0000","last_undegraded":"2024-09-10T10:01:50.332304+0000","last_fullsized":"2024-09-10T10:01:50.332304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.520549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710348+0000","last_change":"2024-09-10T10:01:34.255210+0000","last_active":"2024-09-10T10:01:50.710348+0000","last_peered":"2024-09-10T10:01:50.710348+0000","last_clean":"2024-09-10T10:01:50.710348+0000","last_became_active":"2024-09-10T10:01:34.254861+0000","last_became_peered":"2024-09-10T10:01:34.254861+0000","last_unstale":"2024-09-10T10:01:50.710348+0000","last_undegraded":"2024-09-10T10:01:50.710348+0000","last_fullsized":"2024-09-10T10:01:50.710348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:10.332758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337852+0000","last_change":"2024-09-10T10:01:32.204622+0000","last_active":"2024-09-10T10:01:50.337852+0000","last_peered":"2024-09-10T10:01:50.337852+0000","last_clean":"2024-09-10T10:01:50.337852+0000","last_became_active":"2024-09-10T10:01:32.204236+0000","last_became_peered":"2024-09-10T10:01:32.204236+0000","last_unstale":"2024-09-10T10:01:50.337852+0000","last_undegraded":"2024-09-10T10:01:50.337852+0000","last_fullsized":"2024-09-10T10:01:50.337852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:59.402854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320679+0000","last_change":"2024-09-10T10:01:34.244790+0000","last_active":"2024-09-10T10:01:50.320679+0000","last_peered":"2024-09-10T10:01:50.320679+0000","last_clean":"2024-09-10T10:01:50.320679+0000","last_became_active":"2024-09-10T10:01:34.244443+0000","last_became_peered":"2024-09-10T10:01:34.244443+0000","last_unstale":"2024-09-10T10:01:50.320679+0000","last_undegraded":"2024-09-10T10:01:50.320679+0000","last_fullsized":"2024-09-10T10:01:50.320679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:55.913475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.529774+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.529410+0000","last_became_peered":"2024-09-10T10:01:32.529410+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:01.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335478+0000","last_change":"2024-09-10T10:01:50.335478+0000","last_active":"2024-09-10T10:01:50.335478+0000","last_peered":"2024-09-10T10:01:50.335478+0000","last_clean":"2024-09-10T10:01:50.335478+0000","last_became_active":"2024-09-10T10:01:50.332726+0000","last_became_peered":"2024-09-10T10:01:50.332726+0000","last_unstale":"2024-09-10T10:01:50.335478+0000","last_undegraded":"2024-09-10T10:01:50.335478+0000","last_fullsized":"2024-09-10T10:01:50.335478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:04:10.726066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338880+0000","last_change":"2024-09-10T10:01:32.530401+0000","last_active":"2024-09-10T10:01:50.338880+0000","last_peered":"2024-09-10T10:01:50.338880+0000","last_clean":"2024-09-10T10:01:50.338880+0000","last_became_active":"2024-09-10T10:01:32.529928+0000","last_became_peered":"2024-09-10T10:01:32.529928+0000","last_unstale":"2024-09-10T10:01:50.338880+0000","last_undegraded":"2024-09-10T10:01:50.338880+0000","last_fullsized":"2024-09-10T10:01:50.338880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:18.978923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332474+0000","last_change":"2024-09-10T10:01:34.245777+0000","last_active":"2024-09-10T10:01:50.332474+0000","last_peered":"2024-09-10T10:01:50.332474+0000","last_clean":"2024-09-10T10:01:50.332474+0000","last_became_active":"2024-09-10T10:01:34.245431+0000","last_became_peered":"2024-09-10T10:01:34.245431+0000","last_unstale":"2024-09-10T10:01:50.332474+0000","last_undegraded":"2024-09-10T10:01:50.332474+0000","last_fullsized":"2024-09-10T10:01:50.332474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:17.754117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317896+0000","last_change":"2024-09-10T10:01:32.201916+0000","last_active":"2024-09-10T10:01:50.317896+0000","last_peered":"2024-09-10T10:01:50.317896+0000","last_clean":"2024-09-10T10:01:50.317896+0000","last_became_active":"2024-09-10T10:01:32.201540+0000","last_became_peered":"2024-09-10T10:01:32.201540+0000","last_unstale":"2024-09-10T10:01:50.317896+0000","last_undegraded":"2024-09-10T10:01:50.317896+0000","last_fullsized":"2024-09-10T10:01:50.317896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:41.862673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320509+0000","last_change":"2024-09-10T10:01:34.251334+0000","last_active":"2024-09-10T10:01:50.320509+0000","last_peered":"2024-09-10T10:01:50.320509+0000","last_clean":"2024-09-10T10:01:50.320509+0000","last_became_active":"2024-09-10T10:01:34.251032+0000","last_became_peered":"2024-09-10T10:01:34.251032+0000","last_unstale":"2024-09-10T10:01:50.320509+0000","last_undegraded":"2024-09-10T10:01:50.320509+0000","last_fullsized":"2024-09-10T10:01:50.320509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:26.437259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.839313+0000","last_change":"2024-09-10T10:01:32.255074+0000","last_active":"2024-09-10T10:01:50.839313+0000","last_peered":"2024-09-10T10:01:50.839313+0000","last_clean":"2024-09-10T10:01:50.839313+0000","last_became_active":"2024-09-10T10:01:32.254637+0000","last_became_peered":"2024-09-10T10:01:32.254637+0000","last_unstale":"2024-09-10T10:01:50.839313+0000","last_undegraded":"2024-09-10T10:01:50.839313+0000","last_fullsized":"2024-09-10T10:01:50.839313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:35.392816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321297+0000","last_change":"2024-09-10T10:01:34.253770+0000","last_active":"2024-09-10T10:01:50.321297+0000","last_peered":"2024-09-10T10:01:50.321297+0000","last_clean":"2024-09-10T10:01:50.321297+0000","last_became_active":"2024-09-10T10:01:34.253395+0000","last_became_peered":"2024-09-10T10:01:34.253395+0000","last_unstale":"2024-09-10T10:01:50.321297+0000","last_undegraded":"2024-09-10T10:01:50.321297+0000","last_fullsized":"2024-09-10T10:01:50.321297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:55.264734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337074+0000","last_change":"2024-09-10T10:01:50.337074+0000","last_active":"2024-09-10T10:01:50.337074+0000","last_peered":"2024-09-10T10:01:50.337074+0000","last_clean":"2024-09-10T10:01:50.337074+0000","last_became_active":"2024-09-10T10:01:50.336661+0000","last_became_peered":"2024-09-10T10:01:50.336661+0000","last_unstale":"2024-09-10T10:01:50.337074+0000","last_undegraded":"2024-09-10T10:01:50.337074+0000","last_fullsized":"2024-09-10T10:01:50.337074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:47.700516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338318+0000","last_change":"2024-09-10T10:01:34.249342+0000","last_active":"2024-09-10T10:01:50.338318+0000","last_peered":"2024-09-10T10:01:50.338318+0000","last_clean":"2024-09-10T10:01:50.338318+0000","last_became_active":"2024-09-10T10:01:34.245289+0000","last_became_peered":"2024-09-10T10:01:34.245289+0000","last_unstale":"2024-09-10T10:01:50.338318+0000","last_undegraded":"2024-09-10T10:01:50.338318+0000","last_fullsized":"2024-09-10T10:01:50.338318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:34.068515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320070+0000","last_change":"2024-09-10T10:01:32.199112+0000","last_active":"2024-09-10T10:01:50.320070+0000","last_peered":"2024-09-10T10:01:50.320070+0000","last_clean":"2024-09-10T10:01:50.320070+0000","last_became_active":"2024-09-10T10:01:32.198627+0000","last_became_peered":"2024-09-10T10:01:32.198627+0000","last_unstale":"2024-09-10T10:01:50.320070+0000","last_undegraded":"2024-09-10T10:01:50.320070+0000","last_fullsized":"2024-09-10T10:01:50.320070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:37.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335789+0000","last_change":"2024-09-10T10:01:34.253537+0000","last_active":"2024-09-10T10:01:50.335789+0000","last_peered":"2024-09-10T10:01:50.335789+0000","last_clean":"2024-09-10T10:01:50.335789+0000","last_became_active":"2024-09-10T10:01:34.253060+0000","last_became_peered":"2024-09-10T10:01:34.253060+0000","last_unstale":"2024-09-10T10:01:50.335789+0000","last_undegraded":"2024-09-10T10:01:50.335789+0000","last_fullsized":"2024-09-10T10:01:50.335789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.411289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333010+0000","last_change":"2024-09-10T10:01:32.202595+0000","last_active":"2024-09-10T10:01:50.333010+0000","last_peered":"2024-09-10T10:01:50.333010+0000","last_clean":"2024-09-10T10:01:50.333010+0000","last_became_active":"2024-09-10T10:01:32.201796+0000","last_became_peered":"2024-09-10T10:01:32.201796+0000","last_unstale":"2024-09-10T10:01:50.333010+0000","last_undegraded":"2024-09-10T10:01:50.333010+0000","last_fullsized":"2024-09-10T10:01:50.333010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:54.778442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320621+0000","last_change":"2024-09-10T10:01:34.244734+0000","last_active":"2024-09-10T10:01:50.320621+0000","last_peered":"2024-09-10T10:01:50.320621+0000","last_clean":"2024-09-10T10:01:50.320621+0000","last_became_active":"2024-09-10T10:01:34.244396+0000","last_became_peered":"2024-09-10T10:01:34.244396+0000","last_unstale":"2024-09-10T10:01:50.320621+0000","last_undegraded":"2024-09-10T10:01:50.320621+0000","last_fullsized":"2024-09-10T10:01:50.320621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:02.266670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338190+0000","last_change":"2024-09-10T10:01:32.207322+0000","last_active":"2024-09-10T10:01:50.338190+0000","last_peered":"2024-09-10T10:01:50.338190+0000","last_clean":"2024-09-10T10:01:50.338190+0000","last_became_active":"2024-09-10T10:01:32.206936+0000","last_became_peered":"2024-09-10T10:01:32.206936+0000","last_unstale":"2024-09-10T10:01:50.338190+0000","last_undegraded":"2024-09-10T10:01:50.338190+0000","last_fullsized":"2024-09-10T10:01:50.338190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:17.194749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337740+0000","last_change":"2024-09-10T10:01:34.255626+0000","last_active":"2024-09-10T10:01:50.337740+0000","last_peered":"2024-09-10T10:01:50.337740+0000","last_clean":"2024-09-10T10:01:50.337740+0000","last_became_active":"2024-09-10T10:01:34.255333+0000","last_became_peered":"2024-09-10T10:01:34.255333+0000","last_unstale":"2024-09-10T10:01:50.337740+0000","last_undegraded":"2024-09-10T10:01:50.337740+0000","last_fullsized":"2024-09-10T10:01:50.337740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:26.287576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710505+0000","last_change":"2024-09-10T10:01:34.257209+0000","last_active":"2024-09-10T10:01:50.710505+0000","last_peered":"2024-09-10T10:01:50.710505+0000","last_clean":"2024-09-10T10:01:50.710505+0000","last_became_active":"2024-09-10T10:01:34.256891+0000","last_became_peered":"2024-09-10T10:01:34.256891+0000","last_unstale":"2024-09-10T10:01:50.710505+0000","last_undegraded":"2024-09-10T10:01:50.710505+0000","last_fullsized":"2024-09-10T10:01:50.710505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:58.178876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338664+0000","last_change":"2024-09-10T10:01:32.200462+0000","last_active":"2024-09-10T10:01:50.338664+0000","last_peered":"2024-09-10T10:01:50.338664+0000","last_clean":"2024-09-10T10:01:50.338664+0000","last_became_active":"2024-09-10T10:01:32.198970+0000","last_became_peered":"2024-09-10T10:01:32.198970+0000","last_unstale":"2024-09-10T10:01:50.338664+0000","last_undegraded":"2024-09-10T10:01:50.338664+0000","last_fullsized":"2024-09-10T10:01:50.338664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:00.980894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318326+0000","last_change":"2024-09-10T10:01:34.246984+0000","last_active":"2024-09-10T10:01:50.318326+0000","last_peered":"2024-09-10T10:01:50.318326+0000","last_clean":"2024-09-10T10:01:50.318326+0000","last_became_active":"2024-09-10T10:01:34.246244+0000","last_became_peered":"2024-09-10T10:01:34.246244+0000","last_unstale":"2024-09-10T10:01:50.318326+0000","last_undegraded":"2024-09-10T10:01:50.318326+0000","last_fullsized":"2024-09-10T10:01:50.318326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:47.656959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.342048+0000","last_change":"2024-09-10T10:01:50.341594+0000","last_active":"2024-09-10T10:01:50.342048+0000","last_peered":"2024-09-10T10:01:50.342048+0000","last_clean":"2024-09-10T10:01:50.342048+0000","last_became_active":"2024-09-10T10:01:50.338152+0000","last_became_peered":"2024-09-10T10:01:50.338152+0000","last_unstale":"2024-09-10T10:01:50.342048+0000","last_undegraded":"2024-09-10T10:01:50.342048+0000","last_fullsized":"2024-09-10T10:01:50.342048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:25.484321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320803+0000","last_change":"2024-09-10T10:01:34.246396+0000","last_active":"2024-09-10T10:01:50.320803+0000","last_peered":"2024-09-10T10:01:50.320803+0000","last_clean":"2024-09-10T10:01:50.320803+0000","last_became_active":"2024-09-10T10:01:34.245956+0000","last_became_peered":"2024-09-10T10:01:34.245956+0000","last_unstale":"2024-09-10T10:01:50.320803+0000","last_undegraded":"2024-09-10T10:01:50.320803+0000","last_fullsized":"2024-09-10T10:01:50.320803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:53.440984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320437+0000","last_change":"2024-09-10T10:01:32.529907+0000","last_active":"2024-09-10T10:01:50.320437+0000","last_peered":"2024-09-10T10:01:50.320437+0000","last_clean":"2024-09-10T10:01:50.320437+0000","last_became_active":"2024-09-10T10:01:32.529353+0000","last_became_peered":"2024-09-10T10:01:32.529353+0000","last_unstale":"2024-09-10T10:01:50.320437+0000","last_undegraded":"2024-09-10T10:01:50.320437+0000","last_fullsized":"2024-09-10T10:01:50.320437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:45.562270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320201+0000","last_change":"2024-09-10T10:01:34.243623+0000","last_active":"2024-09-10T10:01:50.320201+0000","last_peered":"2024-09-10T10:01:50.320201+0000","last_clean":"2024-09-10T10:01:50.320201+0000","last_became_active":"2024-09-10T10:01:34.241625+0000","last_became_peered":"2024-09-10T10:01:34.241625+0000","last_unstale":"2024-09-10T10:01:50.320201+0000","last_undegraded":"2024-09-10T10:01:50.320201+0000","last_fullsized":"2024-09-10T10:01:50.320201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:25.112837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.205514+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.205127+0000","last_became_peered":"2024-09-10T10:01:32.205127+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.207240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41863,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":32,"num_read_kb":58,"num_write":66,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":125589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":437776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1552384,"data_stored":1069520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939673,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":438475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168607,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40968,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704184,"statfs":{"total":95995035648,"available":95953084416,"internally_reserved":0,"allocated":1495040,"data_stored":1028843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561065,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1503232,"data_stored":1028443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":437798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215160,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":905216,"data_stored":437997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":946176,"data_stored":478875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":942080,"data_stored":478841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":41099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":41259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:01:57.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:01:58.013 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:58.387 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:58 smithi045 ceph-mon[31114]: pgmap v287: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.9 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:58.388 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:01:58 smithi045 ceph-mon[31114]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:01:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.586+0000 7ffb8b45d640 1 -- 172.21.15.45:0/1253105959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 msgr2=0x7ffb841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.586+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/1253105959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb841021c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009900 tx=0x7ffb8002a410 comp rx=0 tx=0).stop 2024-09-10T10:01:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.586+0000 7ffb8b45d640 1 -- 172.21.15.45:0/1253105959 shutdown_connections 2024-09-10T10:01:58.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.586+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/1253105959 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb84106f10 0x7ffb8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.586+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/1253105959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb84102790 0x7ffb841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.586+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/1253105959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb841021c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 -- 172.21.15.45:0/1253105959 >> 172.21.15.45:0/1253105959 conn(0x7ffb840fd880 msgr2=0x7ffb840ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 -- 172.21.15.45:0/1253105959 shutdown_connections 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 -- 172.21.15.45:0/1253105959 wait complete. 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 Processor -- start 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 -- start start 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb8419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb84102790 0x7ffb8419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb84106f10 0x7ffb8419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84111fe0 con 0x7ffb84101de0 2024-09-10T10:01:58.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84111e60 con 0x7ffb84102790 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.587+0000 7ffb8b45d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffb84112160 con 0x7ffb84106f10 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb8419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb8419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34856/0 (socket says 172.21.15.45:34856) 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 -- 172.21.15.45:0/823560965 learned_addr learned my addr 172.21.15.45:0/823560965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb889f8640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb84102790 0x7ffb8419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb899fa640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb84106f10 0x7ffb8419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 -- 172.21.15.45:0/823560965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb84106f10 msgr2=0x7ffb8419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb84106f10 0x7ffb8419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 -- 172.21.15.45:0/823560965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb84102790 msgr2=0x7ffb8419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb84102790 0x7ffb8419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 -- 172.21.15.45:0/823560965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8419f640 con 0x7ffb84101de0 2024-09-10T10:01:58.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb899fa640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb84106f10 0x7ffb8419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb891f9640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb8419a690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb80004770 tx=0x7ffb80003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb7a7fc640 1 -- 172.21.15.45:0/823560965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8003f070 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb7a7fc640 1 -- 172.21.15.45:0/823560965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb80031ca0 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8419f8d0 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb7a7fc640 1 -- 172.21.15.45:0/823560965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80035030 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.588+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb841a7210 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.589+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb841032c0 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.591+0000 7ffb7a7fc640 1 -- 172.21.15.45:0/823560965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffb80042070 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.591+0000 7ffb7a7fc640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffb700a2c10 0x7ffb700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.591+0000 7ffb889f8640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffb700a2c10 0x7ffb700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.591+0000 7ffb7a7fc640 1 -- 172.21.15.45:0/823560965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ffb800efd80 con 0x7ffb84101de0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.592+0000 7ffb889f8640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffb700a2c10 0x7ffb700a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffb840ff6c0 tx=0x7ffb7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:58.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.594+0000 7ffb7a7fc640 1 -- 172.21.15.45:0/823560965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb800bcca0 con 0x7ffb84101de0 2024-09-10T10:01:58.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:58 smithi162 ceph-mon[34963]: pgmap v287: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.9 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:58.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:01:58 smithi162 ceph-mon[34963]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:01:58.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:58 smithi047 ceph-mon[35244]: pgmap v287: 97 pgs: 97 active+clean; 618 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.9 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-09-10T10:01:58.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:01:58 smithi047 ceph-mon[35244]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:01:58.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.770+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb84075430 con 0x7ffb700a2c10 2024-09-10T10:01:58.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.776+0000 7ffb7a7fc640 1 -- 172.21.15.45:0/823560965 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258640 (secure 0 0 0) 0x7ffb84075430 con 0x7ffb700a2c10 2024-09-10T10:01:58.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:01:58.782 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-10T10:01:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffb700a2c10 msgr2=0x7ffb700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffb700a2c10 0x7ffb700a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffb840ff6c0 tx=0x7ffb7400b410 comp rx=0 tx=0).stop 2024-09-10T10:01:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 msgr2=0x7ffb8419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:58.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb8419a690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb80004770 tx=0x7ffb80003c10 comp rx=0 tx=0).stop 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 shutdown_connections 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffb700a2c10 0x7ffb700a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffb84106f10 0x7ffb8419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb84102790 0x7ffb8419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 --2- 172.21.15.45:0/823560965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb84101de0 0x7ffb8419a690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 >> 172.21.15.45:0/823560965 conn(0x7ffb840fd880 msgr2=0x7ffb840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 shutdown_connections 2024-09-10T10:01:58.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:01:58.780+0000 7ffb8b45d640 1 -- 172.21.15.45:0/823560965 wait complete. 2024-09-10T10:01:59.597 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-10T10:01:58.619166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644567,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":179,"num_read_kb":194,"num_write":262,"num_write_kb":3016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357584,"kb_used_data":12468,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584240,"statfs":{"total":1151940427776,"available":1151574261760,"internally_reserved":0,"allocated":12767232,"data_stored":7187013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45187,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":55,"num_write":31,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.321391"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335498+0000","last_change":"2024-09-10T10:01:34.253963+0000","last_active":"2024-09-10T10:01:50.335498+0000","last_peered":"2024-09-10T10:01:50.335498+0000","last_clean":"2024-09-10T10:01:50.335498+0000","last_became_active":"2024-09-10T10:01:34.252547+0000","last_became_peered":"2024-09-10T10:01:34.252547+0000","last_unstale":"2024-09-10T10:01:50.335498+0000","last_undegraded":"2024-09-10T10:01:50.335498+0000","last_fullsized":"2024-09-10T10:01:50.335498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:55.761035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321219+0000","last_change":"2024-09-10T10:01:34.249329+0000","last_active":"2024-09-10T10:01:50.321219+0000","last_peered":"2024-09-10T10:01:50.321219+0000","last_clean":"2024-09-10T10:01:50.321219+0000","last_became_active":"2024-09-10T10:01:34.249036+0000","last_became_peered":"2024-09-10T10:01:34.249036+0000","last_unstale":"2024-09-10T10:01:50.321219+0000","last_undegraded":"2024-09-10T10:01:50.321219+0000","last_fullsized":"2024-09-10T10:01:50.321219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:26.244307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320430+0000","last_change":"2024-09-10T10:01:34.242770+0000","last_active":"2024-09-10T10:01:50.320430+0000","last_peered":"2024-09-10T10:01:50.320430+0000","last_clean":"2024-09-10T10:01:50.320430+0000","last_became_active":"2024-09-10T10:01:34.242465+0000","last_became_peered":"2024-09-10T10:01:34.242465+0000","last_unstale":"2024-09-10T10:01:50.320430+0000","last_undegraded":"2024-09-10T10:01:50.320430+0000","last_fullsized":"2024-09-10T10:01:50.320430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:32.776914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838909+0000","last_change":"2024-09-10T10:01:34.249110+0000","last_active":"2024-09-10T10:01:50.838909+0000","last_peered":"2024-09-10T10:01:50.838909+0000","last_clean":"2024-09-10T10:01:50.838909+0000","last_became_active":"2024-09-10T10:01:34.248758+0000","last_became_peered":"2024-09-10T10:01:34.248758+0000","last_unstale":"2024-09-10T10:01:50.838909+0000","last_undegraded":"2024-09-10T10:01:50.838909+0000","last_fullsized":"2024-09-10T10:01:50.838909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:51.622686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338095+0000","last_change":"2024-09-10T10:01:34.248799+0000","last_active":"2024-09-10T10:01:50.338095+0000","last_peered":"2024-09-10T10:01:50.338095+0000","last_clean":"2024-09-10T10:01:50.338095+0000","last_became_active":"2024-09-10T10:01:34.246797+0000","last_became_peered":"2024-09-10T10:01:34.246797+0000","last_unstale":"2024-09-10T10:01:50.338095+0000","last_undegraded":"2024-09-10T10:01:50.338095+0000","last_fullsized":"2024-09-10T10:01:50.338095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:33.887465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320541+0000","last_change":"2024-09-10T10:01:34.252139+0000","last_active":"2024-09-10T10:01:50.320541+0000","last_peered":"2024-09-10T10:01:50.320541+0000","last_clean":"2024-09-10T10:01:50.320541+0000","last_became_active":"2024-09-10T10:01:34.251823+0000","last_became_peered":"2024-09-10T10:01:34.251823+0000","last_unstale":"2024-09-10T10:01:50.320541+0000","last_undegraded":"2024-09-10T10:01:50.320541+0000","last_fullsized":"2024-09-10T10:01:50.320541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:31.642428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337590+0000","last_change":"2024-09-10T10:01:34.258097+0000","last_active":"2024-09-10T10:01:50.337590+0000","last_peered":"2024-09-10T10:01:50.337590+0000","last_clean":"2024-09-10T10:01:50.337590+0000","last_became_active":"2024-09-10T10:01:34.257717+0000","last_became_peered":"2024-09-10T10:01:34.257717+0000","last_unstale":"2024-09-10T10:01:50.337590+0000","last_undegraded":"2024-09-10T10:01:50.337590+0000","last_fullsized":"2024-09-10T10:01:50.337590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:16.280503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319894+0000","last_change":"2024-09-10T10:01:34.249901+0000","last_active":"2024-09-10T10:01:50.319894+0000","last_peered":"2024-09-10T10:01:50.319894+0000","last_clean":"2024-09-10T10:01:50.319894+0000","last_became_active":"2024-09-10T10:01:34.249361+0000","last_became_peered":"2024-09-10T10:01:34.249361+0000","last_unstale":"2024-09-10T10:01:50.319894+0000","last_undegraded":"2024-09-10T10:01:50.319894+0000","last_fullsized":"2024-09-10T10:01:50.319894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:42.282621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.711941+0000","last_change":"2024-09-10T10:01:50.711491+0000","last_active":"2024-09-10T10:01:50.711941+0000","last_peered":"2024-09-10T10:01:50.711941+0000","last_clean":"2024-09-10T10:01:50.711941+0000","last_became_active":"2024-09-10T10:01:50.711162+0000","last_became_peered":"2024-09-10T10:01:50.711162+0000","last_unstale":"2024-09-10T10:01:50.711941+0000","last_undegraded":"2024-09-10T10:01:50.711941+0000","last_fullsized":"2024-09-10T10:01:50.711941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:08.502816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320934+0000","last_change":"2024-09-10T10:01:34.252538+0000","last_active":"2024-09-10T10:01:50.320934+0000","last_peered":"2024-09-10T10:01:50.320934+0000","last_clean":"2024-09-10T10:01:50.320934+0000","last_became_active":"2024-09-10T10:01:34.251384+0000","last_became_peered":"2024-09-10T10:01:34.251384+0000","last_unstale":"2024-09-10T10:01:50.320934+0000","last_undegraded":"2024-09-10T10:01:50.320934+0000","last_fullsized":"2024-09-10T10:01:50.320934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:39.990996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338061+0000","last_change":"2024-09-10T10:01:34.254262+0000","last_active":"2024-09-10T10:01:50.338061+0000","last_peered":"2024-09-10T10:01:50.338061+0000","last_clean":"2024-09-10T10:01:50.338061+0000","last_became_active":"2024-09-10T10:01:34.253907+0000","last_became_peered":"2024-09-10T10:01:34.253907+0000","last_unstale":"2024-09-10T10:01:50.338061+0000","last_undegraded":"2024-09-10T10:01:50.338061+0000","last_fullsized":"2024-09-10T10:01:50.338061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:38.681367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838921+0000","last_change":"2024-09-10T10:01:34.252194+0000","last_active":"2024-09-10T10:01:50.838921+0000","last_peered":"2024-09-10T10:01:50.838921+0000","last_clean":"2024-09-10T10:01:50.838921+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.838921+0000","last_undegraded":"2024-09-10T10:01:50.838921+0000","last_fullsized":"2024-09-10T10:01:50.838921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:47.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332280+0000","last_change":"2024-09-10T10:01:34.251795+0000","last_active":"2024-09-10T10:01:50.332280+0000","last_peered":"2024-09-10T10:01:50.332280+0000","last_clean":"2024-09-10T10:01:50.332280+0000","last_became_active":"2024-09-10T10:01:34.251403+0000","last_became_peered":"2024-09-10T10:01:34.251403+0000","last_unstale":"2024-09-10T10:01:50.332280+0000","last_undegraded":"2024-09-10T10:01:50.332280+0000","last_fullsized":"2024-09-10T10:01:50.332280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:11.650068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321079+0000","last_change":"2024-09-10T10:01:34.252718+0000","last_active":"2024-09-10T10:01:50.321079+0000","last_peered":"2024-09-10T10:01:50.321079+0000","last_clean":"2024-09-10T10:01:50.321079+0000","last_became_active":"2024-09-10T10:01:34.252415+0000","last_became_peered":"2024-09-10T10:01:34.252415+0000","last_unstale":"2024-09-10T10:01:50.321079+0000","last_undegraded":"2024-09-10T10:01:50.321079+0000","last_fullsized":"2024-09-10T10:01:50.321079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:51.057191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320828+0000","last_change":"2024-09-10T10:01:34.252237+0000","last_active":"2024-09-10T10:01:50.320828+0000","last_peered":"2024-09-10T10:01:50.320828+0000","last_clean":"2024-09-10T10:01:50.320828+0000","last_became_active":"2024-09-10T10:01:34.251909+0000","last_became_peered":"2024-09-10T10:01:34.251909+0000","last_unstale":"2024-09-10T10:01:50.320828+0000","last_undegraded":"2024-09-10T10:01:50.320828+0000","last_fullsized":"2024-09-10T10:01:50.320828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:15.150503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318205+0000","last_change":"2024-09-10T10:01:34.259107+0000","last_active":"2024-09-10T10:01:50.318205+0000","last_peered":"2024-09-10T10:01:50.318205+0000","last_clean":"2024-09-10T10:01:50.318205+0000","last_became_active":"2024-09-10T10:01:34.258230+0000","last_became_peered":"2024-09-10T10:01:34.258230+0000","last_unstale":"2024-09-10T10:01:50.318205+0000","last_undegraded":"2024-09-10T10:01:50.318205+0000","last_fullsized":"2024-09-10T10:01:50.318205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:36.091358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335573+0000","last_change":"2024-09-10T10:01:34.254341+0000","last_active":"2024-09-10T10:01:50.335573+0000","last_peered":"2024-09-10T10:01:50.335573+0000","last_clean":"2024-09-10T10:01:50.335573+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.335573+0000","last_undegraded":"2024-09-10T10:01:50.335573+0000","last_fullsized":"2024-09-10T10:01:50.335573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:46.668715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317729+0000","last_change":"2024-09-10T10:01:34.247950+0000","last_active":"2024-09-10T10:01:50.317729+0000","last_peered":"2024-09-10T10:01:50.317729+0000","last_clean":"2024-09-10T10:01:50.317729+0000","last_became_active":"2024-09-10T10:01:34.247659+0000","last_became_peered":"2024-09-10T10:01:34.247659+0000","last_unstale":"2024-09-10T10:01:50.317729+0000","last_undegraded":"2024-09-10T10:01:50.317729+0000","last_fullsized":"2024-09-10T10:01:50.317729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:34.113605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333284+0000","last_change":"2024-09-10T10:01:34.248153+0000","last_active":"2024-09-10T10:01:50.333284+0000","last_peered":"2024-09-10T10:01:50.333284+0000","last_clean":"2024-09-10T10:01:50.333284+0000","last_became_active":"2024-09-10T10:01:34.247773+0000","last_became_peered":"2024-09-10T10:01:34.247773+0000","last_unstale":"2024-09-10T10:01:50.333284+0000","last_undegraded":"2024-09-10T10:01:50.333284+0000","last_fullsized":"2024-09-10T10:01:50.333284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:54.223519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335273+0000","last_change":"2024-09-10T10:01:34.252781+0000","last_active":"2024-09-10T10:01:50.335273+0000","last_peered":"2024-09-10T10:01:50.335273+0000","last_clean":"2024-09-10T10:01:50.335273+0000","last_became_active":"2024-09-10T10:01:34.252459+0000","last_became_peered":"2024-09-10T10:01:34.252459+0000","last_unstale":"2024-09-10T10:01:50.335273+0000","last_undegraded":"2024-09-10T10:01:50.335273+0000","last_fullsized":"2024-09-10T10:01:50.335273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:32.039769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332286+0000","last_change":"2024-09-10T10:01:34.251147+0000","last_active":"2024-09-10T10:01:50.332286+0000","last_peered":"2024-09-10T10:01:50.332286+0000","last_clean":"2024-09-10T10:01:50.332286+0000","last_became_active":"2024-09-10T10:01:34.250835+0000","last_became_peered":"2024-09-10T10:01:34.250835+0000","last_unstale":"2024-09-10T10:01:50.332286+0000","last_undegraded":"2024-09-10T10:01:50.332286+0000","last_fullsized":"2024-09-10T10:01:50.332286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:56.914798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318417+0000","last_change":"2024-09-10T10:01:34.259300+0000","last_active":"2024-09-10T10:01:50.318417+0000","last_peered":"2024-09-10T10:01:50.318417+0000","last_clean":"2024-09-10T10:01:50.318417+0000","last_became_active":"2024-09-10T10:01:34.258192+0000","last_became_peered":"2024-09-10T10:01:34.258192+0000","last_unstale":"2024-09-10T10:01:50.318417+0000","last_undegraded":"2024-09-10T10:01:50.318417+0000","last_fullsized":"2024-09-10T10:01:50.318417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:19.755696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319962+0000","last_change":"2024-09-10T10:01:34.244797+0000","last_active":"2024-09-10T10:01:50.319962+0000","last_peered":"2024-09-10T10:01:50.319962+0000","last_clean":"2024-09-10T10:01:50.319962+0000","last_became_active":"2024-09-10T10:01:34.244464+0000","last_became_peered":"2024-09-10T10:01:34.244464+0000","last_unstale":"2024-09-10T10:01:50.319962+0000","last_undegraded":"2024-09-10T10:01:50.319962+0000","last_fullsized":"2024-09-10T10:01:50.319962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:23.750063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710213+0000","last_change":"2024-09-10T10:01:34.248445+0000","last_active":"2024-09-10T10:01:50.710213+0000","last_peered":"2024-09-10T10:01:50.710213+0000","last_clean":"2024-09-10T10:01:50.710213+0000","last_became_active":"2024-09-10T10:01:34.248020+0000","last_became_peered":"2024-09-10T10:01:34.248020+0000","last_unstale":"2024-09-10T10:01:50.710213+0000","last_undegraded":"2024-09-10T10:01:50.710213+0000","last_fullsized":"2024-09-10T10:01:50.710213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:32.096061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339780+0000","last_change":"2024-09-10T10:01:50.339780+0000","last_active":"2024-09-10T10:01:50.339780+0000","last_peered":"2024-09-10T10:01:50.339780+0000","last_clean":"2024-09-10T10:01:50.339780+0000","last_became_active":"2024-09-10T10:01:50.339519+0000","last_became_peered":"2024-09-10T10:01:50.339519+0000","last_unstale":"2024-09-10T10:01:50.339780+0000","last_undegraded":"2024-09-10T10:01:50.339780+0000","last_fullsized":"2024-09-10T10:01:50.339780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:27.368296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333351+0000","last_change":"2024-09-10T10:01:34.244512+0000","last_active":"2024-09-10T10:01:50.333351+0000","last_peered":"2024-09-10T10:01:50.333351+0000","last_clean":"2024-09-10T10:01:50.333351+0000","last_became_active":"2024-09-10T10:01:34.244191+0000","last_became_peered":"2024-09-10T10:01:34.244191+0000","last_unstale":"2024-09-10T10:01:50.333351+0000","last_undegraded":"2024-09-10T10:01:50.333351+0000","last_fullsized":"2024-09-10T10:01:50.333351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:19.019159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335673+0000","last_change":"2024-09-10T10:01:34.248683+0000","last_active":"2024-09-10T10:01:50.335673+0000","last_peered":"2024-09-10T10:01:50.335673+0000","last_clean":"2024-09-10T10:01:50.335673+0000","last_became_active":"2024-09-10T10:01:34.248228+0000","last_became_peered":"2024-09-10T10:01:34.248228+0000","last_unstale":"2024-09-10T10:01:50.335673+0000","last_undegraded":"2024-09-10T10:01:50.335673+0000","last_fullsized":"2024-09-10T10:01:50.335673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:00.485621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332452+0000","last_change":"2024-09-10T10:01:34.254683+0000","last_active":"2024-09-10T10:01:50.332452+0000","last_peered":"2024-09-10T10:01:50.332452+0000","last_clean":"2024-09-10T10:01:50.332452+0000","last_became_active":"2024-09-10T10:01:34.254384+0000","last_became_peered":"2024-09-10T10:01:34.254384+0000","last_unstale":"2024-09-10T10:01:50.332452+0000","last_undegraded":"2024-09-10T10:01:50.332452+0000","last_fullsized":"2024-09-10T10:01:50.332452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:35.404584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321448+0000","last_change":"2024-09-10T10:01:34.254079+0000","last_active":"2024-09-10T10:01:50.321448+0000","last_peered":"2024-09-10T10:01:50.321448+0000","last_clean":"2024-09-10T10:01:50.321448+0000","last_became_active":"2024-09-10T10:01:34.253527+0000","last_became_peered":"2024-09-10T10:01:34.253527+0000","last_unstale":"2024-09-10T10:01:50.321448+0000","last_undegraded":"2024-09-10T10:01:50.321448+0000","last_fullsized":"2024-09-10T10:01:50.321448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:31.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332948+0000","last_change":"2024-09-10T10:01:34.244840+0000","last_active":"2024-09-10T10:01:50.332948+0000","last_peered":"2024-09-10T10:01:50.332948+0000","last_clean":"2024-09-10T10:01:50.332948+0000","last_became_active":"2024-09-10T10:01:34.244482+0000","last_became_peered":"2024-09-10T10:01:34.244482+0000","last_unstale":"2024-09-10T10:01:50.332948+0000","last_undegraded":"2024-09-10T10:01:50.332948+0000","last_fullsized":"2024-09-10T10:01:50.332948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:50.666840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332939+0000","last_change":"2024-09-10T10:01:34.250789+0000","last_active":"2024-09-10T10:01:50.332939+0000","last_peered":"2024-09-10T10:01:50.332939+0000","last_clean":"2024-09-10T10:01:50.332939+0000","last_became_active":"2024-09-10T10:01:34.250445+0000","last_became_peered":"2024-09-10T10:01:34.250445+0000","last_unstale":"2024-09-10T10:01:50.332939+0000","last_undegraded":"2024-09-10T10:01:50.332939+0000","last_fullsized":"2024-09-10T10:01:50.332939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:00.389643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335618+0000","last_change":"2024-09-10T10:01:34.253139+0000","last_active":"2024-09-10T10:01:50.335618+0000","last_peered":"2024-09-10T10:01:50.335618+0000","last_clean":"2024-09-10T10:01:50.335618+0000","last_became_active":"2024-09-10T10:01:34.251801+0000","last_became_peered":"2024-09-10T10:01:34.251801+0000","last_unstale":"2024-09-10T10:01:50.335618+0000","last_undegraded":"2024-09-10T10:01:50.335618+0000","last_fullsized":"2024-09-10T10:01:50.335618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:33.492291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338268+0000","last_change":"2024-09-10T10:01:34.243669+0000","last_active":"2024-09-10T10:01:50.338268+0000","last_peered":"2024-09-10T10:01:50.338268+0000","last_clean":"2024-09-10T10:01:50.338268+0000","last_became_active":"2024-09-10T10:01:34.243291+0000","last_became_peered":"2024-09-10T10:01:34.243291+0000","last_unstale":"2024-09-10T10:01:50.338268+0000","last_undegraded":"2024-09-10T10:01:50.338268+0000","last_fullsized":"2024-09-10T10:01:50.338268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:47.157495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710729+0000","last_change":"2024-09-10T10:01:32.206317+0000","last_active":"2024-09-10T10:01:50.710729+0000","last_peered":"2024-09-10T10:01:50.710729+0000","last_clean":"2024-09-10T10:01:50.710729+0000","last_became_active":"2024-09-10T10:01:32.205412+0000","last_became_peered":"2024-09-10T10:01:32.205412+0000","last_unstale":"2024-09-10T10:01:50.710729+0000","last_undegraded":"2024-09-10T10:01:50.710729+0000","last_fullsized":"2024-09-10T10:01:50.710729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:07.489923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710763+0000","last_change":"2024-09-10T10:01:34.254783+0000","last_active":"2024-09-10T10:01:50.710763+0000","last_peered":"2024-09-10T10:01:50.710763+0000","last_clean":"2024-09-10T10:01:50.710763+0000","last_became_active":"2024-09-10T10:01:34.254465+0000","last_became_peered":"2024-09-10T10:01:34.254465+0000","last_unstale":"2024-09-10T10:01:50.710763+0000","last_undegraded":"2024-09-10T10:01:50.710763+0000","last_fullsized":"2024-09-10T10:01:50.710763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:57.983265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320605+0000","last_change":"2024-09-10T10:01:32.204551+0000","last_active":"2024-09-10T10:01:50.320605+0000","last_peered":"2024-09-10T10:01:50.320605+0000","last_clean":"2024-09-10T10:01:50.320605+0000","last_became_active":"2024-09-10T10:01:32.204214+0000","last_became_peered":"2024-09-10T10:01:32.204214+0000","last_unstale":"2024-09-10T10:01:50.320605+0000","last_undegraded":"2024-09-10T10:01:50.320605+0000","last_fullsized":"2024-09-10T10:01:50.320605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:50.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332156+0000","last_change":"2024-09-10T10:01:32.205907+0000","last_active":"2024-09-10T10:01:50.332156+0000","last_peered":"2024-09-10T10:01:50.332156+0000","last_clean":"2024-09-10T10:01:50.332156+0000","last_became_active":"2024-09-10T10:01:32.205523+0000","last_became_peered":"2024-09-10T10:01:32.205523+0000","last_unstale":"2024-09-10T10:01:50.332156+0000","last_undegraded":"2024-09-10T10:01:50.332156+0000","last_fullsized":"2024-09-10T10:01:50.332156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:14.540234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319983+0000","last_change":"2024-09-10T10:01:34.249826+0000","last_active":"2024-09-10T10:01:50.319983+0000","last_peered":"2024-09-10T10:01:50.319983+0000","last_clean":"2024-09-10T10:01:50.319983+0000","last_became_active":"2024-09-10T10:01:34.248940+0000","last_became_peered":"2024-09-10T10:01:34.248940+0000","last_unstale":"2024-09-10T10:01:50.319983+0000","last_undegraded":"2024-09-10T10:01:50.319983+0000","last_fullsized":"2024-09-10T10:01:50.319983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:49.125909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710836+0000","last_change":"2024-09-10T10:01:32.205559+0000","last_active":"2024-09-10T10:01:50.710836+0000","last_peered":"2024-09-10T10:01:50.710836+0000","last_clean":"2024-09-10T10:01:50.710836+0000","last_became_active":"2024-09-10T10:01:32.205251+0000","last_became_peered":"2024-09-10T10:01:32.205251+0000","last_unstale":"2024-09-10T10:01:50.710836+0000","last_undegraded":"2024-09-10T10:01:50.710836+0000","last_fullsized":"2024-09-10T10:01:50.710836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:11.442234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317525+0000","last_change":"2024-09-10T10:01:34.245852+0000","last_active":"2024-09-10T10:01:50.317525+0000","last_peered":"2024-09-10T10:01:50.317525+0000","last_clean":"2024-09-10T10:01:50.317525+0000","last_became_active":"2024-09-10T10:01:34.245487+0000","last_became_peered":"2024-09-10T10:01:34.245487+0000","last_unstale":"2024-09-10T10:01:50.317525+0000","last_undegraded":"2024-09-10T10:01:50.317525+0000","last_fullsized":"2024-09-10T10:01:50.317525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:01.995443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320839+0000","last_change":"2024-09-10T10:01:32.202754+0000","last_active":"2024-09-10T10:01:50.320839+0000","last_peered":"2024-09-10T10:01:50.320839+0000","last_clean":"2024-09-10T10:01:50.320839+0000","last_became_active":"2024-09-10T10:01:32.202360+0000","last_became_peered":"2024-09-10T10:01:32.202360+0000","last_unstale":"2024-09-10T10:01:50.320839+0000","last_undegraded":"2024-09-10T10:01:50.320839+0000","last_fullsized":"2024-09-10T10:01:50.320839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:04.864276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318627+0000","last_change":"2024-09-10T10:01:34.245670+0000","last_active":"2024-09-10T10:01:50.318627+0000","last_peered":"2024-09-10T10:01:50.318627+0000","last_clean":"2024-09-10T10:01:50.318627+0000","last_became_active":"2024-09-10T10:01:34.244922+0000","last_became_peered":"2024-09-10T10:01:34.244922+0000","last_unstale":"2024-09-10T10:01:50.318627+0000","last_undegraded":"2024-09-10T10:01:50.318627+0000","last_fullsized":"2024-09-10T10:01:50.318627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:05.442144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338108+0000","last_change":"2024-09-10T10:01:32.204658+0000","last_active":"2024-09-10T10:01:50.338108+0000","last_peered":"2024-09-10T10:01:50.338108+0000","last_clean":"2024-09-10T10:01:50.338108+0000","last_became_active":"2024-09-10T10:01:32.204310+0000","last_became_peered":"2024-09-10T10:01:32.204310+0000","last_unstale":"2024-09-10T10:01:50.338108+0000","last_undegraded":"2024-09-10T10:01:50.338108+0000","last_fullsized":"2024-09-10T10:01:50.338108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:16.446977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.712239+0000","last_change":"2024-09-10T10:01:50.712239+0000","last_active":"2024-09-10T10:01:50.712239+0000","last_peered":"2024-09-10T10:01:50.712239+0000","last_clean":"2024-09-10T10:01:50.712239+0000","last_became_active":"2024-09-10T10:01:50.711826+0000","last_became_peered":"2024-09-10T10:01:50.711826+0000","last_unstale":"2024-09-10T10:01:50.712239+0000","last_undegraded":"2024-09-10T10:01:50.712239+0000","last_fullsized":"2024-09-10T10:01:50.712239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:54.670359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710657+0000","last_change":"2024-09-10T10:01:32.530649+0000","last_active":"2024-09-10T10:01:50.710657+0000","last_peered":"2024-09-10T10:01:50.710657+0000","last_clean":"2024-09-10T10:01:50.710657+0000","last_became_active":"2024-09-10T10:01:32.530157+0000","last_became_peered":"2024-09-10T10:01:32.530157+0000","last_unstale":"2024-09-10T10:01:50.710657+0000","last_undegraded":"2024-09-10T10:01:50.710657+0000","last_fullsized":"2024-09-10T10:01:50.710657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:30.235163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339461+0000","last_change":"2024-09-10T10:01:50.339461+0000","last_active":"2024-09-10T10:01:50.339461+0000","last_peered":"2024-09-10T10:01:50.339461+0000","last_clean":"2024-09-10T10:01:50.339461+0000","last_became_active":"2024-09-10T10:01:50.339158+0000","last_became_peered":"2024-09-10T10:01:50.339158+0000","last_unstale":"2024-09-10T10:01:50.339461+0000","last_undegraded":"2024-09-10T10:01:50.339461+0000","last_fullsized":"2024-09-10T10:01:50.339461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:11.733302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'25","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.303430+0000","last_change":"2024-09-10T10:01:32.205933+0000","last_active":"2024-09-10T10:01:56.303430+0000","last_peered":"2024-09-10T10:01:56.303430+0000","last_clean":"2024-09-10T10:01:56.303430+0000","last_became_active":"2024-09-10T10:01:32.205592+0000","last_became_peered":"2024-09-10T10:01:32.205592+0000","last_unstale":"2024-09-10T10:01:56.303430+0000","last_undegraded":"2024-09-10T10:01:56.303430+0000","last_fullsized":"2024-09-10T10:01:56.303430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:29.107388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53167,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":37,"num_read_kb":69,"num_write":25,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338800+0000","last_change":"2024-09-10T10:01:34.249720+0000","last_active":"2024-09-10T10:01:50.338800+0000","last_peered":"2024-09-10T10:01:50.338800+0000","last_clean":"2024-09-10T10:01:50.338800+0000","last_became_active":"2024-09-10T10:01:34.249285+0000","last_became_peered":"2024-09-10T10:01:34.249285+0000","last_unstale":"2024-09-10T10:01:50.338800+0000","last_undegraded":"2024-09-10T10:01:50.338800+0000","last_fullsized":"2024-09-10T10:01:50.338800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:30.787344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333124+0000","last_change":"2024-09-10T10:01:32.202622+0000","last_active":"2024-09-10T10:01:50.333124+0000","last_peered":"2024-09-10T10:01:50.333124+0000","last_clean":"2024-09-10T10:01:50.333124+0000","last_became_active":"2024-09-10T10:01:32.200870+0000","last_became_peered":"2024-09-10T10:01:32.200870+0000","last_unstale":"2024-09-10T10:01:50.333124+0000","last_undegraded":"2024-09-10T10:01:50.333124+0000","last_fullsized":"2024-09-10T10:01:50.333124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:48.562971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332189+0000","last_change":"2024-09-10T10:01:34.252622+0000","last_active":"2024-09-10T10:01:50.332189+0000","last_peered":"2024-09-10T10:01:50.332189+0000","last_clean":"2024-09-10T10:01:50.332189+0000","last_became_active":"2024-09-10T10:01:34.252271+0000","last_became_peered":"2024-09-10T10:01:34.252271+0000","last_unstale":"2024-09-10T10:01:50.332189+0000","last_undegraded":"2024-09-10T10:01:50.332189+0000","last_fullsized":"2024-09-10T10:01:50.332189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:03.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710592+0000","last_change":"2024-09-10T10:01:32.206133+0000","last_active":"2024-09-10T10:01:50.710592+0000","last_peered":"2024-09-10T10:01:50.710592+0000","last_clean":"2024-09-10T10:01:50.710592+0000","last_became_active":"2024-09-10T10:01:32.205747+0000","last_became_peered":"2024-09-10T10:01:32.205747+0000","last_unstale":"2024-09-10T10:01:50.710592+0000","last_undegraded":"2024-09-10T10:01:50.710592+0000","last_fullsized":"2024-09-10T10:01:50.710592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:30.515614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337830+0000","last_change":"2024-09-10T10:01:34.257702+0000","last_active":"2024-09-10T10:01:50.337830+0000","last_peered":"2024-09-10T10:01:50.337830+0000","last_clean":"2024-09-10T10:01:50.337830+0000","last_became_active":"2024-09-10T10:01:34.257366+0000","last_became_peered":"2024-09-10T10:01:34.257366+0000","last_unstale":"2024-09-10T10:01:50.337830+0000","last_undegraded":"2024-09-10T10:01:50.337830+0000","last_fullsized":"2024-09-10T10:01:50.337830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:51.772522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838725+0000","last_change":"2024-09-10T10:01:32.205527+0000","last_active":"2024-09-10T10:01:50.838725+0000","last_peered":"2024-09-10T10:01:50.838725+0000","last_clean":"2024-09-10T10:01:50.838725+0000","last_became_active":"2024-09-10T10:01:32.205162+0000","last_became_peered":"2024-09-10T10:01:32.205162+0000","last_unstale":"2024-09-10T10:01:50.838725+0000","last_undegraded":"2024-09-10T10:01:50.838725+0000","last_fullsized":"2024-09-10T10:01:50.838725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:28.149331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337435+0000","last_change":"2024-09-10T10:01:34.250691+0000","last_active":"2024-09-10T10:01:50.337435+0000","last_peered":"2024-09-10T10:01:50.337435+0000","last_clean":"2024-09-10T10:01:50.337435+0000","last_became_active":"2024-09-10T10:01:34.249303+0000","last_became_peered":"2024-09-10T10:01:34.249303+0000","last_unstale":"2024-09-10T10:01:50.337435+0000","last_undegraded":"2024-09-10T10:01:50.337435+0000","last_fullsized":"2024-09-10T10:01:50.337435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:47.413567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338071+0000","last_change":"2024-09-10T10:01:32.530043+0000","last_active":"2024-09-10T10:01:50.338071+0000","last_peered":"2024-09-10T10:01:50.338071+0000","last_clean":"2024-09-10T10:01:50.338071+0000","last_became_active":"2024-09-10T10:01:32.529502+0000","last_became_peered":"2024-09-10T10:01:32.529502+0000","last_unstale":"2024-09-10T10:01:50.338071+0000","last_undegraded":"2024-09-10T10:01:50.338071+0000","last_fullsized":"2024-09-10T10:01:50.338071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:26.133771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710304+0000","last_change":"2024-09-10T10:01:34.254722+0000","last_active":"2024-09-10T10:01:50.710304+0000","last_peered":"2024-09-10T10:01:50.710304+0000","last_clean":"2024-09-10T10:01:50.710304+0000","last_became_active":"2024-09-10T10:01:34.254390+0000","last_became_peered":"2024-09-10T10:01:34.254390+0000","last_unstale":"2024-09-10T10:01:50.710304+0000","last_undegraded":"2024-09-10T10:01:50.710304+0000","last_fullsized":"2024-09-10T10:01:50.710304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:37.261256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339061+0000","last_change":"2024-09-10T10:01:32.200744+0000","last_active":"2024-09-10T10:01:50.339061+0000","last_peered":"2024-09-10T10:01:50.339061+0000","last_clean":"2024-09-10T10:01:50.339061+0000","last_became_active":"2024-09-10T10:01:32.199906+0000","last_became_peered":"2024-09-10T10:01:32.199906+0000","last_unstale":"2024-09-10T10:01:50.339061+0000","last_undegraded":"2024-09-10T10:01:50.339061+0000","last_fullsized":"2024-09-10T10:01:50.339061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:17.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.840135+0000","last_change":"2024-09-10T10:01:50.840135+0000","last_active":"2024-09-10T10:01:50.840135+0000","last_peered":"2024-09-10T10:01:50.840135+0000","last_clean":"2024-09-10T10:01:50.840135+0000","last_became_active":"2024-09-10T10:01:50.839693+0000","last_became_peered":"2024-09-10T10:01:50.839693+0000","last_unstale":"2024-09-10T10:01:50.840135+0000","last_undegraded":"2024-09-10T10:01:50.840135+0000","last_fullsized":"2024-09-10T10:01:50.840135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:01.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332542+0000","last_change":"2024-09-10T10:01:32.200383+0000","last_active":"2024-09-10T10:01:50.332542+0000","last_peered":"2024-09-10T10:01:50.332542+0000","last_clean":"2024-09-10T10:01:50.332542+0000","last_became_active":"2024-09-10T10:01:32.199926+0000","last_became_peered":"2024-09-10T10:01:32.199926+0000","last_unstale":"2024-09-10T10:01:50.332542+0000","last_undegraded":"2024-09-10T10:01:50.332542+0000","last_fullsized":"2024-09-10T10:01:50.332542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:29.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338221+0000","last_change":"2024-09-10T10:01:34.250077+0000","last_active":"2024-09-10T10:01:50.338221+0000","last_peered":"2024-09-10T10:01:50.338221+0000","last_clean":"2024-09-10T10:01:50.338221+0000","last_became_active":"2024-09-10T10:01:34.249697+0000","last_became_peered":"2024-09-10T10:01:34.249697+0000","last_unstale":"2024-09-10T10:01:50.338221+0000","last_undegraded":"2024-09-10T10:01:50.338221+0000","last_fullsized":"2024-09-10T10:01:50.338221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:25.068115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710730+0000","last_change":"2024-09-10T10:01:32.200235+0000","last_active":"2024-09-10T10:01:50.710730+0000","last_peered":"2024-09-10T10:01:50.710730+0000","last_clean":"2024-09-10T10:01:50.710730+0000","last_became_active":"2024-09-10T10:01:32.199920+0000","last_became_peered":"2024-09-10T10:01:32.199920+0000","last_unstale":"2024-09-10T10:01:50.710730+0000","last_undegraded":"2024-09-10T10:01:50.710730+0000","last_fullsized":"2024-09-10T10:01:50.710730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:10.980054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338054+0000","last_change":"2024-09-10T10:01:34.256937+0000","last_active":"2024-09-10T10:01:50.338054+0000","last_peered":"2024-09-10T10:01:50.338054+0000","last_clean":"2024-09-10T10:01:50.338054+0000","last_became_active":"2024-09-10T10:01:34.256376+0000","last_became_peered":"2024-09-10T10:01:34.256376+0000","last_unstale":"2024-09-10T10:01:50.338054+0000","last_undegraded":"2024-09-10T10:01:50.338054+0000","last_fullsized":"2024-09-10T10:01:50.338054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:07.386683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333436+0000","last_change":"2024-09-10T10:01:32.530794+0000","last_active":"2024-09-10T10:01:50.333436+0000","last_peered":"2024-09-10T10:01:50.333436+0000","last_clean":"2024-09-10T10:01:50.333436+0000","last_became_active":"2024-09-10T10:01:32.530335+0000","last_became_peered":"2024-09-10T10:01:32.530335+0000","last_unstale":"2024-09-10T10:01:50.333436+0000","last_undegraded":"2024-09-10T10:01:50.333436+0000","last_fullsized":"2024-09-10T10:01:50.333436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:40.883029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338123+0000","last_change":"2024-09-10T10:01:34.250228+0000","last_active":"2024-09-10T10:01:50.338123+0000","last_peered":"2024-09-10T10:01:50.338123+0000","last_clean":"2024-09-10T10:01:50.338123+0000","last_became_active":"2024-09-10T10:01:34.249901+0000","last_became_peered":"2024-09-10T10:01:34.249901+0000","last_unstale":"2024-09-10T10:01:50.338123+0000","last_undegraded":"2024-09-10T10:01:50.338123+0000","last_fullsized":"2024-09-10T10:01:50.338123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:10.710185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337584+0000","last_change":"2024-09-10T10:00:23.346149+0000","last_active":"2024-09-10T10:01:50.337584+0000","last_peered":"2024-09-10T10:01:50.337584+0000","last_clean":"2024-09-10T10:01:50.337584+0000","last_became_active":"2024-09-10T10:00:23.345608+0000","last_became_peered":"2024-09-10T10:00:23.345608+0000","last_unstale":"2024-09-10T10:01:50.337584+0000","last_undegraded":"2024-09-10T10:01:50.337584+0000","last_fullsized":"2024-09-10T10:01:50.337584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_clean_scrub_stamp":"2024-09-10T09:57:03.194756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:14.092838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838755+0000","last_change":"2024-09-10T10:01:32.200726+0000","last_active":"2024-09-10T10:01:50.838755+0000","last_peered":"2024-09-10T10:01:50.838755+0000","last_clean":"2024-09-10T10:01:50.838755+0000","last_became_active":"2024-09-10T10:01:32.200163+0000","last_became_peered":"2024-09-10T10:01:32.200163+0000","last_unstale":"2024-09-10T10:01:50.838755+0000","last_undegraded":"2024-09-10T10:01:50.838755+0000","last_fullsized":"2024-09-10T10:01:50.838755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:05.300476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320505+0000","last_change":"2024-09-10T10:01:34.243258+0000","last_active":"2024-09-10T10:01:50.320505+0000","last_peered":"2024-09-10T10:01:50.320505+0000","last_clean":"2024-09-10T10:01:50.320505+0000","last_became_active":"2024-09-10T10:01:34.242060+0000","last_became_peered":"2024-09-10T10:01:34.242060+0000","last_unstale":"2024-09-10T10:01:50.320505+0000","last_undegraded":"2024-09-10T10:01:50.320505+0000","last_fullsized":"2024-09-10T10:01:50.320505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:02.910538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337582+0000","last_change":"2024-09-10T10:01:32.205174+0000","last_active":"2024-09-10T10:01:50.337582+0000","last_peered":"2024-09-10T10:01:50.337582+0000","last_clean":"2024-09-10T10:01:50.337582+0000","last_became_active":"2024-09-10T10:01:32.204121+0000","last_became_peered":"2024-09-10T10:01:32.204121+0000","last_unstale":"2024-09-10T10:01:50.337582+0000","last_undegraded":"2024-09-10T10:01:50.337582+0000","last_fullsized":"2024-09-10T10:01:50.337582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:42.533591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332204+0000","last_change":"2024-09-10T10:01:34.254895+0000","last_active":"2024-09-10T10:01:50.332204+0000","last_peered":"2024-09-10T10:01:50.332204+0000","last_clean":"2024-09-10T10:01:50.332204+0000","last_became_active":"2024-09-10T10:01:34.254572+0000","last_became_peered":"2024-09-10T10:01:34.254572+0000","last_unstale":"2024-09-10T10:01:50.332204+0000","last_undegraded":"2024-09-10T10:01:50.332204+0000","last_fullsized":"2024-09-10T10:01:50.332204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:44.194941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332304+0000","last_change":"2024-09-10T10:01:32.530093+0000","last_active":"2024-09-10T10:01:50.332304+0000","last_peered":"2024-09-10T10:01:50.332304+0000","last_clean":"2024-09-10T10:01:50.332304+0000","last_became_active":"2024-09-10T10:01:32.529756+0000","last_became_peered":"2024-09-10T10:01:32.529756+0000","last_unstale":"2024-09-10T10:01:50.332304+0000","last_undegraded":"2024-09-10T10:01:50.332304+0000","last_fullsized":"2024-09-10T10:01:50.332304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.520549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710348+0000","last_change":"2024-09-10T10:01:34.255210+0000","last_active":"2024-09-10T10:01:50.710348+0000","last_peered":"2024-09-10T10:01:50.710348+0000","last_clean":"2024-09-10T10:01:50.710348+0000","last_became_active":"2024-09-10T10:01:34.254861+0000","last_became_peered":"2024-09-10T10:01:34.254861+0000","last_unstale":"2024-09-10T10:01:50.710348+0000","last_undegraded":"2024-09-10T10:01:50.710348+0000","last_fullsized":"2024-09-10T10:01:50.710348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:10.332758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337852+0000","last_change":"2024-09-10T10:01:32.204622+0000","last_active":"2024-09-10T10:01:50.337852+0000","last_peered":"2024-09-10T10:01:50.337852+0000","last_clean":"2024-09-10T10:01:50.337852+0000","last_became_active":"2024-09-10T10:01:32.204236+0000","last_became_peered":"2024-09-10T10:01:32.204236+0000","last_unstale":"2024-09-10T10:01:50.337852+0000","last_undegraded":"2024-09-10T10:01:50.337852+0000","last_fullsized":"2024-09-10T10:01:50.337852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:59.402854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320679+0000","last_change":"2024-09-10T10:01:34.244790+0000","last_active":"2024-09-10T10:01:50.320679+0000","last_peered":"2024-09-10T10:01:50.320679+0000","last_clean":"2024-09-10T10:01:50.320679+0000","last_became_active":"2024-09-10T10:01:34.244443+0000","last_became_peered":"2024-09-10T10:01:34.244443+0000","last_unstale":"2024-09-10T10:01:50.320679+0000","last_undegraded":"2024-09-10T10:01:50.320679+0000","last_fullsized":"2024-09-10T10:01:50.320679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:55.913475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.529774+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.529410+0000","last_became_peered":"2024-09-10T10:01:32.529410+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:01.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335478+0000","last_change":"2024-09-10T10:01:50.335478+0000","last_active":"2024-09-10T10:01:50.335478+0000","last_peered":"2024-09-10T10:01:50.335478+0000","last_clean":"2024-09-10T10:01:50.335478+0000","last_became_active":"2024-09-10T10:01:50.332726+0000","last_became_peered":"2024-09-10T10:01:50.332726+0000","last_unstale":"2024-09-10T10:01:50.335478+0000","last_undegraded":"2024-09-10T10:01:50.335478+0000","last_fullsized":"2024-09-10T10:01:50.335478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:04:10.726066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338880+0000","last_change":"2024-09-10T10:01:32.530401+0000","last_active":"2024-09-10T10:01:50.338880+0000","last_peered":"2024-09-10T10:01:50.338880+0000","last_clean":"2024-09-10T10:01:50.338880+0000","last_became_active":"2024-09-10T10:01:32.529928+0000","last_became_peered":"2024-09-10T10:01:32.529928+0000","last_unstale":"2024-09-10T10:01:50.338880+0000","last_undegraded":"2024-09-10T10:01:50.338880+0000","last_fullsized":"2024-09-10T10:01:50.338880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:18.978923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332474+0000","last_change":"2024-09-10T10:01:34.245777+0000","last_active":"2024-09-10T10:01:50.332474+0000","last_peered":"2024-09-10T10:01:50.332474+0000","last_clean":"2024-09-10T10:01:50.332474+0000","last_became_active":"2024-09-10T10:01:34.245431+0000","last_became_peered":"2024-09-10T10:01:34.245431+0000","last_unstale":"2024-09-10T10:01:50.332474+0000","last_undegraded":"2024-09-10T10:01:50.332474+0000","last_fullsized":"2024-09-10T10:01:50.332474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:17.754117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:55.246733+0000","last_change":"2024-09-10T10:01:32.201916+0000","last_active":"2024-09-10T10:01:55.246733+0000","last_peered":"2024-09-10T10:01:55.246733+0000","last_clean":"2024-09-10T10:01:55.246733+0000","last_became_active":"2024-09-10T10:01:32.201540+0000","last_became_peered":"2024-09-10T10:01:32.201540+0000","last_unstale":"2024-09-10T10:01:55.246733+0000","last_undegraded":"2024-09-10T10:01:55.246733+0000","last_fullsized":"2024-09-10T10:01:55.246733+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:41.862673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320509+0000","last_change":"2024-09-10T10:01:34.251334+0000","last_active":"2024-09-10T10:01:50.320509+0000","last_peered":"2024-09-10T10:01:50.320509+0000","last_clean":"2024-09-10T10:01:50.320509+0000","last_became_active":"2024-09-10T10:01:34.251032+0000","last_became_peered":"2024-09-10T10:01:34.251032+0000","last_unstale":"2024-09-10T10:01:50.320509+0000","last_undegraded":"2024-09-10T10:01:50.320509+0000","last_fullsized":"2024-09-10T10:01:50.320509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:26.437259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.839313+0000","last_change":"2024-09-10T10:01:32.255074+0000","last_active":"2024-09-10T10:01:50.839313+0000","last_peered":"2024-09-10T10:01:50.839313+0000","last_clean":"2024-09-10T10:01:50.839313+0000","last_became_active":"2024-09-10T10:01:32.254637+0000","last_became_peered":"2024-09-10T10:01:32.254637+0000","last_unstale":"2024-09-10T10:01:50.839313+0000","last_undegraded":"2024-09-10T10:01:50.839313+0000","last_fullsized":"2024-09-10T10:01:50.839313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:35.392816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321297+0000","last_change":"2024-09-10T10:01:34.253770+0000","last_active":"2024-09-10T10:01:50.321297+0000","last_peered":"2024-09-10T10:01:50.321297+0000","last_clean":"2024-09-10T10:01:50.321297+0000","last_became_active":"2024-09-10T10:01:34.253395+0000","last_became_peered":"2024-09-10T10:01:34.253395+0000","last_unstale":"2024-09-10T10:01:50.321297+0000","last_undegraded":"2024-09-10T10:01:50.321297+0000","last_fullsized":"2024-09-10T10:01:50.321297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:55.264734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337074+0000","last_change":"2024-09-10T10:01:50.337074+0000","last_active":"2024-09-10T10:01:50.337074+0000","last_peered":"2024-09-10T10:01:50.337074+0000","last_clean":"2024-09-10T10:01:50.337074+0000","last_became_active":"2024-09-10T10:01:50.336661+0000","last_became_peered":"2024-09-10T10:01:50.336661+0000","last_unstale":"2024-09-10T10:01:50.337074+0000","last_undegraded":"2024-09-10T10:01:50.337074+0000","last_fullsized":"2024-09-10T10:01:50.337074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:47.700516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338318+0000","last_change":"2024-09-10T10:01:34.249342+0000","last_active":"2024-09-10T10:01:50.338318+0000","last_peered":"2024-09-10T10:01:50.338318+0000","last_clean":"2024-09-10T10:01:50.338318+0000","last_became_active":"2024-09-10T10:01:34.245289+0000","last_became_peered":"2024-09-10T10:01:34.245289+0000","last_unstale":"2024-09-10T10:01:50.338318+0000","last_undegraded":"2024-09-10T10:01:50.338318+0000","last_fullsized":"2024-09-10T10:01:50.338318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:34.068515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320070+0000","last_change":"2024-09-10T10:01:32.199112+0000","last_active":"2024-09-10T10:01:50.320070+0000","last_peered":"2024-09-10T10:01:50.320070+0000","last_clean":"2024-09-10T10:01:50.320070+0000","last_became_active":"2024-09-10T10:01:32.198627+0000","last_became_peered":"2024-09-10T10:01:32.198627+0000","last_unstale":"2024-09-10T10:01:50.320070+0000","last_undegraded":"2024-09-10T10:01:50.320070+0000","last_fullsized":"2024-09-10T10:01:50.320070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:37.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335789+0000","last_change":"2024-09-10T10:01:34.253537+0000","last_active":"2024-09-10T10:01:50.335789+0000","last_peered":"2024-09-10T10:01:50.335789+0000","last_clean":"2024-09-10T10:01:50.335789+0000","last_became_active":"2024-09-10T10:01:34.253060+0000","last_became_peered":"2024-09-10T10:01:34.253060+0000","last_unstale":"2024-09-10T10:01:50.335789+0000","last_undegraded":"2024-09-10T10:01:50.335789+0000","last_fullsized":"2024-09-10T10:01:50.335789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.411289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333010+0000","last_change":"2024-09-10T10:01:32.202595+0000","last_active":"2024-09-10T10:01:50.333010+0000","last_peered":"2024-09-10T10:01:50.333010+0000","last_clean":"2024-09-10T10:01:50.333010+0000","last_became_active":"2024-09-10T10:01:32.201796+0000","last_became_peered":"2024-09-10T10:01:32.201796+0000","last_unstale":"2024-09-10T10:01:50.333010+0000","last_undegraded":"2024-09-10T10:01:50.333010+0000","last_fullsized":"2024-09-10T10:01:50.333010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:54.778442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320621+0000","last_change":"2024-09-10T10:01:34.244734+0000","last_active":"2024-09-10T10:01:50.320621+0000","last_peered":"2024-09-10T10:01:50.320621+0000","last_clean":"2024-09-10T10:01:50.320621+0000","last_became_active":"2024-09-10T10:01:34.244396+0000","last_became_peered":"2024-09-10T10:01:34.244396+0000","last_unstale":"2024-09-10T10:01:50.320621+0000","last_undegraded":"2024-09-10T10:01:50.320621+0000","last_fullsized":"2024-09-10T10:01:50.320621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:02.266670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338190+0000","last_change":"2024-09-10T10:01:32.207322+0000","last_active":"2024-09-10T10:01:50.338190+0000","last_peered":"2024-09-10T10:01:50.338190+0000","last_clean":"2024-09-10T10:01:50.338190+0000","last_became_active":"2024-09-10T10:01:32.206936+0000","last_became_peered":"2024-09-10T10:01:32.206936+0000","last_unstale":"2024-09-10T10:01:50.338190+0000","last_undegraded":"2024-09-10T10:01:50.338190+0000","last_fullsized":"2024-09-10T10:01:50.338190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:17.194749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337740+0000","last_change":"2024-09-10T10:01:34.255626+0000","last_active":"2024-09-10T10:01:50.337740+0000","last_peered":"2024-09-10T10:01:50.337740+0000","last_clean":"2024-09-10T10:01:50.337740+0000","last_became_active":"2024-09-10T10:01:34.255333+0000","last_became_peered":"2024-09-10T10:01:34.255333+0000","last_unstale":"2024-09-10T10:01:50.337740+0000","last_undegraded":"2024-09-10T10:01:50.337740+0000","last_fullsized":"2024-09-10T10:01:50.337740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:26.287576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710505+0000","last_change":"2024-09-10T10:01:34.257209+0000","last_active":"2024-09-10T10:01:50.710505+0000","last_peered":"2024-09-10T10:01:50.710505+0000","last_clean":"2024-09-10T10:01:50.710505+0000","last_became_active":"2024-09-10T10:01:34.256891+0000","last_became_peered":"2024-09-10T10:01:34.256891+0000","last_unstale":"2024-09-10T10:01:50.710505+0000","last_undegraded":"2024-09-10T10:01:50.710505+0000","last_fullsized":"2024-09-10T10:01:50.710505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:58.178876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338664+0000","last_change":"2024-09-10T10:01:32.200462+0000","last_active":"2024-09-10T10:01:50.338664+0000","last_peered":"2024-09-10T10:01:50.338664+0000","last_clean":"2024-09-10T10:01:50.338664+0000","last_became_active":"2024-09-10T10:01:32.198970+0000","last_became_peered":"2024-09-10T10:01:32.198970+0000","last_unstale":"2024-09-10T10:01:50.338664+0000","last_undegraded":"2024-09-10T10:01:50.338664+0000","last_fullsized":"2024-09-10T10:01:50.338664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:00.980894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318326+0000","last_change":"2024-09-10T10:01:34.246984+0000","last_active":"2024-09-10T10:01:50.318326+0000","last_peered":"2024-09-10T10:01:50.318326+0000","last_clean":"2024-09-10T10:01:50.318326+0000","last_became_active":"2024-09-10T10:01:34.246244+0000","last_became_peered":"2024-09-10T10:01:34.246244+0000","last_unstale":"2024-09-10T10:01:50.318326+0000","last_undegraded":"2024-09-10T10:01:50.318326+0000","last_fullsized":"2024-09-10T10:01:50.318326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:47.656959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.342048+0000","last_change":"2024-09-10T10:01:50.341594+0000","last_active":"2024-09-10T10:01:50.342048+0000","last_peered":"2024-09-10T10:01:50.342048+0000","last_clean":"2024-09-10T10:01:50.342048+0000","last_became_active":"2024-09-10T10:01:50.338152+0000","last_became_peered":"2024-09-10T10:01:50.338152+0000","last_unstale":"2024-09-10T10:01:50.342048+0000","last_undegraded":"2024-09-10T10:01:50.342048+0000","last_fullsized":"2024-09-10T10:01:50.342048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:25.484321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320803+0000","last_change":"2024-09-10T10:01:34.246396+0000","last_active":"2024-09-10T10:01:50.320803+0000","last_peered":"2024-09-10T10:01:50.320803+0000","last_clean":"2024-09-10T10:01:50.320803+0000","last_became_active":"2024-09-10T10:01:34.245956+0000","last_became_peered":"2024-09-10T10:01:34.245956+0000","last_unstale":"2024-09-10T10:01:50.320803+0000","last_undegraded":"2024-09-10T10:01:50.320803+0000","last_fullsized":"2024-09-10T10:01:50.320803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:53.440984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320437+0000","last_change":"2024-09-10T10:01:32.529907+0000","last_active":"2024-09-10T10:01:50.320437+0000","last_peered":"2024-09-10T10:01:50.320437+0000","last_clean":"2024-09-10T10:01:50.320437+0000","last_became_active":"2024-09-10T10:01:32.529353+0000","last_became_peered":"2024-09-10T10:01:32.529353+0000","last_unstale":"2024-09-10T10:01:50.320437+0000","last_undegraded":"2024-09-10T10:01:50.320437+0000","last_fullsized":"2024-09-10T10:01:50.320437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:45.562270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320201+0000","last_change":"2024-09-10T10:01:34.243623+0000","last_active":"2024-09-10T10:01:50.320201+0000","last_peered":"2024-09-10T10:01:50.320201+0000","last_clean":"2024-09-10T10:01:50.320201+0000","last_became_active":"2024-09-10T10:01:34.241625+0000","last_became_peered":"2024-09-10T10:01:34.241625+0000","last_unstale":"2024-09-10T10:01:50.320201+0000","last_undegraded":"2024-09-10T10:01:50.320201+0000","last_fullsized":"2024-09-10T10:01:50.320201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:25.112837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.205514+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.205127+0000","last_became_peered":"2024-09-10T10:01:32.205127+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.207240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53965,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":43,"num_read_kb":75,"num_write":79,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":161895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":437776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1564672,"data_stored":1081622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939674,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40968,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704184,"statfs":{"total":95995035648,"available":95953084416,"internally_reserved":0,"allocated":1495040,"data_stored":1028843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561066,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789998,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1503232,"data_stored":1028443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":437798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215161,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":437997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":490977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":490943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":53167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":53201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:01:59.602 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T10:01:59.603 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T10:01:59.603 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T10:01:59.863 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/1890091229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd81037c0 msgr2=0x7f1fd810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/1890091229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd81037c0 0x7f1fd810a050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009ca0 tx=0x7f1fd002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/1890091229 shutdown_connections 2024-09-10T10:02:00.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/1890091229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd81037c0 0x7f1fd810a050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/1890091229 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd8102e00 0x7f1fd8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/1890091229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd8101c00 0x7f1fd8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/1890091229 >> 172.21.15.45:0/1890091229 conn(0x7f1fd80fd390 msgr2=0x7f1fd80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/1890091229 shutdown_connections 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.357+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/1890091229 wait complete. 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 Processor -- start 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 -- start start 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd8101c00 0x7f1fd8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd8102e00 0x7f1fd80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd81037c0 0x7f1fd8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd810d8e0 con 0x7f1fd8101c00 2024-09-10T10:02:00.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd810d760 con 0x7f1fd8102e00 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fdd8e9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd810da60 con 0x7f1fd81037c0 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd8101c00 0x7f1fd8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd8101c00 0x7f1fd8075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34880/0 (socket says 172.21.15.45:34880) 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 -- 172.21.15.45:0/3768603568 learned_addr learned my addr 172.21.15.45:0/3768603568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd77fe640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd81037c0 0x7f1fd8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd67fc640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd8102e00 0x7f1fd80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 -- 172.21.15.45:0/3768603568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd81037c0 msgr2=0x7f1fd8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd81037c0 0x7f1fd8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 -- 172.21.15.45:0/3768603568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd8102e00 msgr2=0x7f1fd80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd8102e00 0x7f1fd80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd6ffd640 1 -- 172.21.15.45:0/3768603568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd80753e0 con 0x7f1fd8101c00 2024-09-10T10:02:00.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.358+0000 7f1fd67fc640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd8102e00 0x7f1fd80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.359+0000 7f1fd6ffd640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd8101c00 0x7f1fd8075f90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800d770 tx=0x7f1fc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.359+0000 7f1fdc8e7640 1 -- 172.21.15.45:0/3768603568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8017070 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.359+0000 7f1fdc8e7640 1 -- 172.21.15.45:0/3768603568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc8009c30 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.359+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd81a2090 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.359+0000 7f1fdc8e7640 1 -- 172.21.15.45:0/3768603568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8003dd0 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.359+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd81a2670 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.360+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd8109000 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.361+0000 7f1fdc8e7640 1 -- 172.21.15.45:0/3768603568 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1fc8003fb0 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.362+0000 7f1fdc8e7640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1fc00a2c90 0x7f1fc00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.362+0000 7f1fd67fc640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1fc00a2c90 0x7f1fc00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.362+0000 7f1fdc8e7640 1 -- 172.21.15.45:0/3768603568 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1fc80cec00 con 0x7f1fd8101c00 2024-09-10T10:02:00.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.362+0000 7f1fd67fc640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1fc00a2c90 0x7f1fc00a5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006fd0 tx=0x7f1fcc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:00.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.366+0000 7f1fdc8e7640 1 -- 172.21.15.45:0/3768603568 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fc8009de0 con 0x7f1fd8101c00 2024-09-10T10:02:00.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.607+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1fd802d7c0 con 0x7f1fd8101c00 2024-09-10T10:02:00.609 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:00 smithi045 ceph-mon[31114]: pgmap v288: 97 pgs: 97 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.9 KiB/s wr, 5 op/s; 12 B/s, 0 objects/s recovering 2024-09-10T10:02:00.609 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:00 smithi045 ceph-mon[31114]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:00.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.610+0000 7f1fdc8e7640 1 -- 172.21.15.45:0/3768603568 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1fc809ba10 con 0x7f1fd8101c00 2024-09-10T10:02:00.611 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-09-10T10:02:00.611 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQAYGeBm3hZOJBAA7tQatxViRe0TfuU6eWQ6uw== 2024-09-10T10:02:00.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1fc00a2c90 msgr2=0x7f1fc00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1fc00a2c90 0x7f1fc00a5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc006fd0 tx=0x7f1fcc008040 comp rx=0 tx=0).stop 2024-09-10T10:02:00.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd8101c00 msgr2=0x7f1fd8075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:00.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd8101c00 0x7f1fd8075f90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800d770 tx=0x7f1fc800dc30 comp rx=0 tx=0).stop 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 shutdown_connections 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1fc00a2c90 0x7f1fc00a5150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd81037c0 0x7f1fd8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd8102e00 0x7f1fd80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 --2- 172.21.15.45:0/3768603568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd8101c00 0x7f1fd8075f90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 >> 172.21.15.45:0/3768603568 conn(0x7f1fd80fd390 msgr2=0x7f1fd81061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 shutdown_connections 2024-09-10T10:02:00.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:00.612+0000 7f1fdd8e9640 1 -- 172.21.15.45:0/3768603568 wait complete. 2024-09-10T10:02:00.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:00 smithi162 ceph-mon[34963]: pgmap v288: 97 pgs: 97 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.9 KiB/s wr, 5 op/s; 12 B/s, 0 objects/s recovering 2024-09-10T10:02:00.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:00 smithi162 ceph-mon[34963]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:00.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:00 smithi047 ceph-mon[35244]: pgmap v288: 97 pgs: 97 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.9 KiB/s wr, 5 op/s; 12 B/s, 0 objects/s recovering 2024-09-10T10:02:00.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:00 smithi047 ceph-mon[35244]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:01.267 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T10:02:01.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T10:02:01.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T10:02:01.306 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T10:02:01.306 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T10:02:01.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T10:02:01.598 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:01 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3768603568' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:02:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:01 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3768603568' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:02:01.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:01 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3768603568' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:02:01.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:01 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3768603568' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:02:01.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:01 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3768603568' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T10:02:01.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:01 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3768603568' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T10:02:02.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 -- 172.21.15.45:0/4039431871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e4410c170 msgr2=0x7f9e4410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/4039431871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e4410c170 0x7f9e4410e610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009ca0 tx=0x7f9e3c02a950 comp rx=0 tx=0).stop 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 -- 172.21.15.45:0/4039431871 shutdown_connections 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/4039431871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e4410c170 0x7f9e4410e610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/4039431871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e44103660 0x7f9e4410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/4039431871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e44102d40 0x7f9e44103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 -- 172.21.15.45:0/4039431871 >> 172.21.15.45:0/4039431871 conn(0x7f9e440fc7a0 msgr2=0x7f9e440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 -- 172.21.15.45:0/4039431871 shutdown_connections 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.071+0000 7f9e4b60d640 1 -- 172.21.15.45:0/4039431871 wait complete. 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 Processor -- start 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 -- start start 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e44102d40 0x7f9e44198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e44103660 0x7f9e44198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e4410c170 0x7f9e4419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44111fb0 con 0x7f9e44103660 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44111e30 con 0x7f9e44102d40 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e4b60d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44112130 con 0x7f9e4410c170 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e48ba8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e44103660 0x7f9e44198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e48ba8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e44103660 0x7f9e44198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34900/0 (socket says 172.21.15.45:34900) 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.072+0000 7f9e48ba8640 1 -- 172.21.15.45:0/177389972 learned_addr learned my addr 172.21.15.45:0/177389972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e49baa640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e4410c170 0x7f9e4419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e493a9640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e44102d40 0x7f9e44198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e48ba8640 1 -- 172.21.15.45:0/177389972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e4410c170 msgr2=0x7f9e4419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e48ba8640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e4410c170 0x7f9e4419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e48ba8640 1 -- 172.21.15.45:0/177389972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e44102d40 msgr2=0x7f9e44198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e48ba8640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e44102d40 0x7f9e44198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e48ba8640 1 -- 172.21.15.45:0/177389972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4419d3e0 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e49baa640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e4410c170 0x7f9e4419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e493a9640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e44102d40 0x7f9e44198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e48ba8640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e44103660 0x7f9e44198970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400b3f0 tx=0x7f9e3400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e3a7fc640 1 -- 172.21.15.45:0/177389972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34015020 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e3a7fc640 1 -- 172.21.15.45:0/177389972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e34011d10 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4419d6d0 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e3a7fc640 1 -- 172.21.15.45:0/177389972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400f2e0 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.073+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e441a4fb0 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.074+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e44103120 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.075+0000 7f9e3a7fc640 1 -- 172.21.15.45:0/177389972 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e34018020 con 0x7f9e44103660 2024-09-10T10:02:02.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.076+0000 7f9e3a7fc640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f9e300a2c10 0x7f9e300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.076+0000 7f9e493a9640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f9e300a2c10 0x7f9e300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.076+0000 7f9e3a7fc640 1 -- 172.21.15.45:0/177389972 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9e340cef60 con 0x7f9e44103660 2024-09-10T10:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.076+0000 7f9e493a9640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f9e300a2c10 0x7f9e300a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e400058c0 tx=0x7f9e40005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.079+0000 7f9e3a7fc640 1 -- 172.21.15.45:0/177389972 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e3401f020 con 0x7f9e44103660 2024-09-10T10:02:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.308+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9e441a5250 con 0x7f9e44103660 2024-09-10T10:02:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.313+0000 7f9e3a7fc640 1 -- 172.21.15.45:0/177389972 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263193 (secure 0 0 0) 0x7f9e3409be80 con 0x7f9e44103660 2024-09-10T10:02:02.317 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f9e300a2c10 msgr2=0x7f9e300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f9e300a2c10 0x7f9e300a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e400058c0 tx=0x7f9e40005850 comp rx=0 tx=0).stop 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e44103660 msgr2=0x7f9e44198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e44103660 0x7f9e44198970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400b3f0 tx=0x7f9e3400b8b0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 shutdown_connections 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f9e300a2c10 0x7f9e300a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9e4410c170 0x7f9e4419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e44103660 0x7f9e44198970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 --2- 172.21.15.45:0/177389972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e44102d40 0x7f9e44198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 >> 172.21.15.45:0/177389972 conn(0x7f9e440fc7a0 msgr2=0x7f9e440fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 shutdown_connections 2024-09-10T10:02:02.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:02.317+0000 7f9e4b60d640 1 -- 172.21.15.45:0/177389972 wait complete. 2024-09-10T10:02:02.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:02 smithi162 ceph-mon[34963]: pgmap v289: 97 pgs: 97 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.3 KiB/s wr, 5 op/s; 11 B/s, 0 objects/s recovering 2024-09-10T10:02:02.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:02 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/177389972' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:02:02.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:02 smithi045 ceph-mon[31114]: pgmap v289: 97 pgs: 97 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.3 KiB/s wr, 5 op/s; 11 B/s, 0 objects/s recovering 2024-09-10T10:02:02.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:02 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/177389972' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:02:02.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:02 smithi047 ceph-mon[35244]: pgmap v289: 97 pgs: 97 active+clean; 629 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.3 KiB/s wr, 5 op/s; 11 B/s, 0 objects/s recovering 2024-09-10T10:02:02.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:02 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/177389972' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T10:02:03.094 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":952575454},{"type":"v1","addr":"172.21.15.45:6801","nonce":952575454}]},"active_addr":"172.21.15.45:6801/952575454","active_change":"2024-09-10T09:54:48.533692+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3649460339}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":21896051}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":689699829}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":384122965}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":375260435}]}]} 2024-09-10T10:02:03.100 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T10:02:03.101 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T10:02:03.101 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:02:03.358 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 -- 172.21.15.45:0/1301696176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54100a80 msgr2=0x7f2a54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 --2- 172.21.15.45:0/1301696176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54100a80 0x7f2a54100ee0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009930 tx=0x7f2a4802a420 comp rx=0 tx=0).stop 2024-09-10T10:02:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 -- 172.21.15.45:0/1301696176 shutdown_connections 2024-09-10T10:02:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 --2- 172.21.15.45:0/1301696176 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a54101420 0x7f2a5410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 --2- 172.21.15.45:0/1301696176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54100a80 0x7f2a54100ee0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 --2- 172.21.15.45:0/1301696176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a54106a80 0x7f2a54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 -- 172.21.15.45:0/1301696176 >> 172.21.15.45:0/1301696176 conn(0x7f2a540fc7a0 msgr2=0x7f2a540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 -- 172.21.15.45:0/1301696176 shutdown_connections 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 -- 172.21.15.45:0/1301696176 wait complete. 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.855+0000 7f2a59694640 1 Processor -- start 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a59694640 1 -- start start 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a59694640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a54100a80 0x7f2a54198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a59694640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a54101420 0x7f2a54198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a59694640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54106a80 0x7f2a5419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a59694640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54110ef0 con 0x7f2a54106a80 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a59694640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54110d70 con 0x7f2a54101420 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a59694640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2a54111070 con 0x7f2a54100a80 2024-09-10T10:02:03.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54106a80 0x7f2a5419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54106a80 0x7f2a5419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47596/0 (socket says 172.21.15.45:47596) 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 -- 172.21.15.45:0/1430384697 learned_addr learned my addr 172.21.15.45:0/1430384697 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a527fc640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a54101420 0x7f2a54198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 -- 172.21.15.45:0/1430384697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a54100a80 msgr2=0x7f2a54198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a54100a80 0x7f2a54198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 -- 172.21.15.45:0/1430384697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a54101420 msgr2=0x7f2a54198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a54101420 0x7f2a54198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.856+0000 7f2a537fe640 1 -- 172.21.15.45:0/1430384697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5419d400 con 0x7f2a54106a80 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.857+0000 7f2a537fe640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54106a80 0x7f2a5419cd20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00b970 tx=0x7f2a4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.857+0000 7f2a3bfff640 1 -- 172.21.15.45:0/1430384697 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c019070 con 0x7f2a54106a80 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.857+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5419d6f0 con 0x7f2a54106a80 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.857+0000 7f2a3bfff640 1 -- 172.21.15.45:0/1430384697 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a4c00cb40 con 0x7f2a54106a80 2024-09-10T10:02:03.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.857+0000 7f2a3bfff640 1 -- 172.21.15.45:0/1430384697 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c007650 con 0x7f2a54106a80 2024-09-10T10:02:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.857+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a541a4fd0 con 0x7f2a54106a80 2024-09-10T10:02:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.858+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a54100ee0 con 0x7f2a54106a80 2024-09-10T10:02:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.859+0000 7f2a3bfff640 1 -- 172.21.15.45:0/1430384697 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2a4c01c070 con 0x7f2a54106a80 2024-09-10T10:02:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.860+0000 7f2a3bfff640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f2a3c0a2c10 0x7f2a3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.860+0000 7f2a52ffd640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f2a3c0a2c10 0x7f2a3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.860+0000 7f2a3bfff640 1 -- 172.21.15.45:0/1430384697 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2a4c0ce6a0 con 0x7f2a54106a80 2024-09-10T10:02:03.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.860+0000 7f2a52ffd640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f2a3c0a2c10 0x7f2a3c0a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400a990 tx=0x7f2a44005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:03.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:03.864+0000 7f2a3bfff640 1 -- 172.21.15.45:0/1430384697 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a4c09af70 con 0x7f2a54106a80 2024-09-10T10:02:04.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.041+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a541a5270 con 0x7f2a54106a80 2024-09-10T10:02:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.042+0000 7f2a3bfff640 1 -- 172.21.15.45:0/1430384697 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22675 (secure 0 0 0) 0x7f2a4c09f270 con 0x7f2a54106a80 2024-09-10T10:02:04.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:04.044 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:01:50.297428+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f2a3c0a2c10 msgr2=0x7f2a3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f2a3c0a2c10 0x7f2a3c0a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400a990 tx=0x7f2a44005c70 comp rx=0 tx=0).stop 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54106a80 msgr2=0x7f2a5419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54106a80 0x7f2a5419cd20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00b970 tx=0x7f2a4c00be30 comp rx=0 tx=0).stop 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 shutdown_connections 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f2a3c0a2c10 0x7f2a3c0a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a54106a80 0x7f2a5419cd20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a54101420 0x7f2a54198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 --2- 172.21.15.45:0/1430384697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2a54100a80 0x7f2a54198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 >> 172.21.15.45:0/1430384697 conn(0x7f2a540fc7a0 msgr2=0x7f2a541047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 shutdown_connections 2024-09-10T10:02:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:04.044+0000 7f2a59694640 1 -- 172.21.15.45:0/1430384697 wait complete. 2024-09-10T10:02:04.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:04 smithi162 ceph-mon[34963]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.3 KiB/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T10:02:04.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:04 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1430384697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:04.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:04 smithi045 ceph-mon[31114]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.3 KiB/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T10:02:04.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:04 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1430384697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:04.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:04 smithi047 ceph-mon[35244]: pgmap v290: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 5.3 KiB/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T10:02:04.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:04 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1430384697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:04.730 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T10:02:04.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:02:04.987 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:05.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 -- 172.21.15.45:0/1913787109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 msgr2=0x7fad4c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 --2- 172.21.15.45:0/1913787109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c10ea90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fad44009ca0 tx=0x7fad4402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 -- 172.21.15.45:0/1913787109 shutdown_connections 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 --2- 172.21.15.45:0/1913787109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c10ea90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 --2- 172.21.15.45:0/1913787109 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad4c102790 0x7fad4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 --2- 172.21.15.45:0/1913787109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad4c101de0 0x7fad4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 -- 172.21.15.45:0/1913787109 >> 172.21.15.45:0/1913787109 conn(0x7fad4c0fd8c0 msgr2=0x7fad4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 -- 172.21.15.45:0/1913787109 shutdown_connections 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.433+0000 7fad51690640 1 -- 172.21.15.45:0/1913787109 wait complete. 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 Processor -- start 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 -- start start 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad4c101de0 0x7fad4c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad4c102790 0x7fad4c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:05.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c112010 con 0x7fad4c106f10 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c111e90 con 0x7fad4c102790 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad51690640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c112190 con 0x7fad4c101de0 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad4b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c19cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47614/0 (socket says 172.21.15.45:47614) 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad4b7fe640 1 -- 172.21.15.45:0/3907150908 learned_addr learned my addr 172.21.15.45:0/3907150908 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.434+0000 7fad4a7fc640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad4c102790 0x7fad4c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad4affd640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad4c101de0 0x7fad4c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad4b7fe640 1 -- 172.21.15.45:0/3907150908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad4c101de0 msgr2=0x7fad4c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad4b7fe640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad4c101de0 0x7fad4c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad4b7fe640 1 -- 172.21.15.45:0/3907150908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad4c102790 msgr2=0x7fad4c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad4b7fe640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad4c102790 0x7fad4c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:05.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad4b7fe640 1 -- 172.21.15.45:0/3907150908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c19d5a0 con 0x7fad4c106f10 2024-09-10T10:02:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad4b7fe640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c19cec0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad44031a60 tx=0x7fad44031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad33fff640 1 -- 172.21.15.45:0/3907150908 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44003b70 con 0x7fad4c106f10 2024-09-10T10:02:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c19d830 con 0x7fad4c106f10 2024-09-10T10:02:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad33fff640 1 -- 172.21.15.45:0/3907150908 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad44003d10 con 0x7fad4c106f10 2024-09-10T10:02:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad33fff640 1 -- 172.21.15.45:0/3907150908 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44004770 con 0x7fad4c106f10 2024-09-10T10:02:05.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.435+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c074420 con 0x7fad4c106f10 2024-09-10T10:02:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.436+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c1032c0 con 0x7fad4c106f10 2024-09-10T10:02:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.437+0000 7fad33fff640 1 -- 172.21.15.45:0/3907150908 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fad4402f790 con 0x7fad4c106f10 2024-09-10T10:02:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.438+0000 7fad33fff640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fad340a2c10 0x7fad340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.438+0000 7fad33fff640 1 -- 172.21.15.45:0/3907150908 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fad440f0400 con 0x7fad4c106f10 2024-09-10T10:02:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.438+0000 7fad4affd640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fad340a2c10 0x7fad340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:05.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.439+0000 7fad4affd640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fad340a2c10 0x7fad340a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009890 tx=0x7fad3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:05.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.442+0000 7fad33fff640 1 -- 172.21.15.45:0/3907150908 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad440bd320 con 0x7fad4c106f10 2024-09-10T10:02:05.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.619+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad4c10a0f0 con 0x7fad4c106f10 2024-09-10T10:02:06.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.621+0000 7fad33fff640 1 -- 172.21.15.45:0/3907150908 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22675 (secure 0 0 0) 0x7fad440bcc20 con 0x7fad4c106f10 2024-09-10T10:02:06.001 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:06.002 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:01:50.297428+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fad340a2c10 msgr2=0x7fad340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fad340a2c10 0x7fad340a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009890 tx=0x7fad3c008040 comp rx=0 tx=0).stop 2024-09-10T10:02:06.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 msgr2=0x7fad4c19cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c19cec0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fad44031a60 tx=0x7fad44031a90 comp rx=0 tx=0).stop 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 shutdown_connections 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fad340a2c10 0x7fad340a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad4c106f10 0x7fad4c19cec0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad4c102790 0x7fad4c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 --2- 172.21.15.45:0/3907150908 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fad4c101de0 0x7fad4c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 >> 172.21.15.45:0/3907150908 conn(0x7fad4c0fd8c0 msgr2=0x7fad4c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 shutdown_connections 2024-09-10T10:02:06.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:05.623+0000 7fad51690640 1 -- 172.21.15.45:0/3907150908 wait complete. 2024-09-10T10:02:06.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T10:02:06.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T10:02:06.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T10:02:06.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T10:02:06.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T10:02:06.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T10:02:06.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T10:02:06.281 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T10:02:06.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T10:02:06.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T10:02:06.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T10:02:06.283 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T10:02:06.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:06 smithi162 ceph-mon[34963]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-10T10:02:06.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:06 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3907150908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:06 smithi045 ceph-mon[31114]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-10T10:02:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:06 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3907150908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:06.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:06 smithi047 ceph-mon[35244]: pgmap v291: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-09-10T10:02:06.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:06 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3907150908' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:06.928 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.266 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.277 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:07.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 -- 172.21.15.45:0/1465090840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 msgr2=0x7fec04107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 --2- 172.21.15.45:0/1465090840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 0x7fec04107b90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7febec009930 tx=0x7febec02a420 comp rx=0 tx=0).stop 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 -- 172.21.15.45:0/1465090840 shutdown_connections 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 --2- 172.21.15.45:0/1465090840 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec041082a0 0x7fec0410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 --2- 172.21.15.45:0/1465090840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 0x7fec04107b90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 --2- 172.21.15.45:0/1465090840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec04102d30 0x7fec04103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 -- 172.21.15.45:0/1465090840 >> 172.21.15.45:0/1465090840 conn(0x7fec040fc7d0 msgr2=0x7fec040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 -- 172.21.15.45:0/1465090840 shutdown_connections 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 -- 172.21.15.45:0/1465090840 wait complete. 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 Processor -- start 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.524+0000 7fec0a324640 1 -- start start 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec0a324640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec04102d30 0x7fec04100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec0a324640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 0x7fec04100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec0a324640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec041082a0 0x7fec04101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec0a324640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec041120a0 con 0x7fec041036e0 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec0a324640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fec04111f20 con 0x7fec04102d30 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec0a324640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fec04112220 con 0x7fec041082a0 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 0x7fec04100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 0x7fec04100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47630/0 (socket says 172.21.15.45:47630) 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 -- 172.21.15.45:0/2991800935 learned_addr learned my addr 172.21.15.45:0/2991800935 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec088c1640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec041082a0 0x7fec04101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec03fff640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec04102d30 0x7fec04100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 -- 172.21.15.45:0/2991800935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec041082a0 msgr2=0x7fec04101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec041082a0 0x7fec04101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 -- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec04102d30 msgr2=0x7fec04100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fec04102d30 0x7fec04100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 -- 172.21.15.45:0/2991800935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec041a6b00 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec088c1640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fec041082a0 0x7fec04101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7febfbfff640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 0x7fec04100ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7febec005940 tx=0x7febec0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec03f030 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.525+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febec0044c0 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.526+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041a6d90 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.526+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec035070 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.526+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a7260 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.526+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec040628c0 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.528+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7febec042030 con 0x7fec041036e0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.528+0000 7fec01ffb640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7febf00a2c90 0x7febf00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec03fff640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7febf00a2c90 0x7febf00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7febec0efc50 con 0x7fec041036e0 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec01ffb640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] conn(0x7febf00ac840 0x7febf00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 --> [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febec03ce50 con 0x7febf00ac840 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7febec0f00a0 con 0x7fec041036e0 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec03fff640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7febf00a2c90 0x7febf00a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febf4009b60 tx=0x7febf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec088c1640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] conn(0x7febf00ac840 0x7febf00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.529+0000 7fec088c1640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] conn(0x7febf00ac840 0x7febf00aece0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.530+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== osd.1 v2:172.21.15.47:6800/3842809154 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febec03ce50 con 0x7febf00ac840 2024-09-10T10:02:07.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.544+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 --> [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec04062cc0 con 0x7febf00ac840 2024-09-10T10:02:07.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec01ffb640 1 -- 172.21.15.45:0/2991800935 <== osd.1 v2:172.21.15.47:6800/3842809154 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec04062cc0 con 0x7febf00ac840 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] conn(0x7febf00ac840 msgr2=0x7febf00aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec0a324640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] conn(0x7febf00ac840 0x7febf00aece0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7febf00a2c90 msgr2=0x7febf00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec0a324640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7febf00a2c90 0x7febf00a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febf4009b60 tx=0x7febf4005f90 comp rx=0 tx=0).stop 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 msgr2=0x7fec04100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec0a324640 1 --2- 172.21.15.45:0/2991800935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec041036e0 0x7fec04100ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7febec005940 tx=0x7febec0040c0 comp rx=0 tx=0).stop 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.545+0000 7fec088c1640 1 -- 172.21.15.45:0/2991800935 reap_dead start 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.546+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 shutdown_connections 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.546+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 >> 172.21.15.45:0/2991800935 conn(0x7fec040fc7d0 msgr2=0x7fec04106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.546+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 shutdown_connections 2024-09-10T10:02:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:07.546+0000 7fec0a324640 1 -- 172.21.15.45:0/2991800935 wait complete. 2024-09-10T10:02:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 -- 172.21.15.45:0/2895522503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538100a80 msgr2=0x7f7538100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 --2- 172.21.15.45:0/2895522503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538100a80 0x7f7538100ee0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f752c009930 tx=0x7f752c02a420 comp rx=0 tx=0).stop 2024-09-10T10:02:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 -- 172.21.15.45:0/2895522503 shutdown_connections 2024-09-10T10:02:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 --2- 172.21.15.45:0/2895522503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7538101420 0x7f753810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 --2- 172.21.15.45:0/2895522503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538100a80 0x7f7538100ee0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 --2- 172.21.15.45:0/2895522503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7538106a80 0x7f7538106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 -- 172.21.15.45:0/2895522503 >> 172.21.15.45:0/2895522503 conn(0x7f75380fc7a0 msgr2=0x7f75380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.314+0000 7f753cc8d640 1 -- 172.21.15.45:0/2895522503 shutdown_connections 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.315+0000 7f753cc8d640 1 -- 172.21.15.45:0/2895522503 wait complete. 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.315+0000 7f753cc8d640 1 Processor -- start 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753cc8d640 1 -- start start 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753cc8d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7538100a80 0x7f7538197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753cc8d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7538101420 0x7f7538197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753cc8d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538106a80 0x7f7538197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753cc8d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7538110e40 con 0x7f7538106a80 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753cc8d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7538110cc0 con 0x7f7538101420 2024-09-10T10:02:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753cc8d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7538110fc0 con 0x7f7538100a80 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538106a80 0x7f7538197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538106a80 0x7f7538197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47640/0 (socket says 172.21.15.45:47640) 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7535d9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7538101420 0x7f7538197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 -- 172.21.15.45:0/2628111014 learned_addr learned my addr 172.21.15.45:0/2628111014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753659c640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7538100a80 0x7f7538197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 -- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7538100a80 msgr2=0x7f7538197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7538100a80 0x7f7538197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 -- 172.21.15.45:0/2628111014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7538101420 msgr2=0x7f7538197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7538101420 0x7f7538197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 -- 172.21.15.45:0/2628111014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75381a6990 con 0x7f7538106a80 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f753659c640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7538100a80 0x7f7538197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7535d9b640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7538101420 0x7f7538197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.316+0000 7f7536d9d640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538106a80 0x7f7538197ba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f753000cad0 tx=0x7f753000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.317+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753000ae20 con 0x7f7538106a80 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.317+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7530005020 con 0x7f7538106a80 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.317+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75381a6c80 con 0x7f7538106a80 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.317+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753000f6f0 con 0x7f7538106a80 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.317+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381a7140 con 0x7f7538106a80 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.318+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7538062940 con 0x7f7538106a80 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.319+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f75300051d0 con 0x7f7538106a80 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f75277fe640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f75200a2c10 0x7f75200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f753659c640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f75200a2c10 0x7f75200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f75300cf5d0 con 0x7f7538106a80 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f75277fe640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] conn(0x7f75200ac7c0 0x7f75200aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 --> [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75200af330 con 0x7f75200ac7c0 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f75300cfa20 con 0x7f7538106a80 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f753659c640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f75200a2c10 0x7f75200a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7528005fd0 tx=0x7f75280094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.320+0000 7f7535d9b640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] conn(0x7f75200ac7c0 0x7f75200aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.321+0000 7f7535d9b640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] conn(0x7f75200ac7c0 0x7f75200aec60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.322+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== osd.8 v2:172.21.15.162:6816/3586576942 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75200af330 con 0x7f75200ac7c0 2024-09-10T10:02:08.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.336+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 --> [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7538062dc0 con 0x7f75200ac7c0 2024-09-10T10:02:08.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.336+0000 7f75277fe640 1 -- 172.21.15.45:0/2628111014 <== osd.8 v2:172.21.15.162:6816/3586576942 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7538062dc0 con 0x7f75200ac7c0 2024-09-10T10:02:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] conn(0x7f75200ac7c0 msgr2=0x7f75200aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] conn(0x7f75200ac7c0 0x7f75200aec60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f75200a2c10 msgr2=0x7f75200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f75200a2c10 0x7f75200a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7528005fd0 tx=0x7f75280094d0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538106a80 msgr2=0x7f7538197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 --2- 172.21.15.45:0/2628111014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7538106a80 0x7f7538197ba0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f753000cad0 tx=0x7f753000cf90 comp rx=0 tx=0).stop 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f7536d9d640 1 -- 172.21.15.45:0/2628111014 reap_dead start 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 shutdown_connections 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 >> 172.21.15.45:0/2628111014 conn(0x7f75380fc7a0 msgr2=0x7f75380fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 shutdown_connections 2024-09-10T10:02:08.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.337+0000 7f753cc8d640 1 -- 172.21.15.45:0/2628111014 wait complete. 2024-09-10T10:02:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 -- 172.21.15.45:0/2123102933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 msgr2=0x7f18c0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 --2- 172.21.15.45:0/2123102933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 0x7f18c0106e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009cd0 tx=0x7f18b002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 -- 172.21.15.45:0/2123102933 shutdown_connections 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 --2- 172.21.15.45:0/2123102933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c0101420 0x7f18c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 --2- 172.21.15.45:0/2123102933 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18c0100a80 0x7f18c0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 --2- 172.21.15.45:0/2123102933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 0x7f18c0106e60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 -- 172.21.15.45:0/2123102933 >> 172.21.15.45:0/2123102933 conn(0x7f18c00fc7a0 msgr2=0x7f18c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 -- 172.21.15.45:0/2123102933 shutdown_connections 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 -- 172.21.15.45:0/2123102933 wait complete. 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.515+0000 7f18c5f7d640 1 Processor -- start 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18c5f7d640 1 -- start start 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18c5f7d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c0100a80 0x7f18c0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18c5f7d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18c0101420 0x7f18c0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18c5f7d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 0x7f18c01a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18c5f7d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0110e80 con 0x7f18c0106a80 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18c5f7d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0110d00 con 0x7f18c0100a80 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18c5f7d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f18c0111000 con 0x7f18c0101420 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 0x7f18c01a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 0x7f18c01a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47660/0 (socket says 172.21.15.45:47660) 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 -- 172.21.15.45:0/497087490 learned_addr learned my addr 172.21.15.45:0/497087490 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18beffd640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18c0101420 0x7f18c0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bf7fe640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c0100a80 0x7f18c0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 -- 172.21.15.45:0/497087490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18c0101420 msgr2=0x7f18c0199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f18c0101420 0x7f18c0199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 -- 172.21.15.45:0/497087490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c0100a80 msgr2=0x7f18c0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c0100a80 0x7f18c0199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 -- 172.21.15.45:0/497087490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c01a4810 con 0x7f18c0106a80 2024-09-10T10:02:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bf7fe640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f18c0100a80 0x7f18c0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.516+0000 7f18bffff640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 0x7f18c01a42d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d950 tx=0x7f18b800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.517+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8017030 con 0x7f18c0106a80 2024-09-10T10:02:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.517+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b8004020 con 0x7f18c0106a80 2024-09-10T10:02:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.517+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c01a49e0 con 0x7f18c0106a80 2024-09-10T10:02:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.517+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8003b70 con 0x7f18c0106a80 2024-09-10T10:02:08.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.517+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01a4ea0 con 0x7f18c0106a80 2024-09-10T10:02:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.518+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18c0062900 con 0x7f18c0106a80 2024-09-10T10:02:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.519+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f18b801a070 con 0x7f18c0106a80 2024-09-10T10:02:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.519+0000 7f18bcff9640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18bf7fe640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f18b80cec80 con 0x7f18c0106a80 2024-09-10T10:02:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18bcff9640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] conn(0x7f18a80ac7c0 0x7f18a80aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 --> [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18a80af330 con 0x7f18a80ac7c0 2024-09-10T10:02:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f18b80d5020 con 0x7f18c0106a80 2024-09-10T10:02:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18beffd640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] conn(0x7f18a80ac7c0 0x7f18a80aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18bf7fe640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f18a80a2c10 0x7f18a80a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f18b0000f80 tx=0x7f18b00093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.520+0000 7f18beffd640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] conn(0x7f18a80ac7c0 0x7f18a80aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:08.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.521+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== osd.0 v2:172.21.15.45:6802/378340133 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18a80af330 con 0x7f18a80ac7c0 2024-09-10T10:02:08.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.534+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 --> [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18c0062db0 con 0x7f18a80ac7c0 2024-09-10T10:02:08.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18bcff9640 1 -- 172.21.15.45:0/497087490 <== osd.0 v2:172.21.15.45:6802/378340133 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f18c0062db0 con 0x7f18a80ac7c0 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] conn(0x7f18a80ac7c0 msgr2=0x7f18a80aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] conn(0x7f18a80ac7c0 0x7f18a80aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f18a80a2c10 msgr2=0x7f18a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f18a80a2c10 0x7f18a80a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f18b0000f80 tx=0x7f18b00093f0 comp rx=0 tx=0).stop 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 msgr2=0x7f18c01a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 --2- 172.21.15.45:0/497087490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18c0106a80 0x7f18c01a42d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d950 tx=0x7f18b800dd50 comp rx=0 tx=0).stop 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18bffff640 1 -- 172.21.15.45:0/497087490 reap_dead start 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 shutdown_connections 2024-09-10T10:02:08.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 >> 172.21.15.45:0/497087490 conn(0x7f18c00fc7a0 msgr2=0x7f18c00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 shutdown_connections 2024-09-10T10:02:08.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:08.535+0000 7f18c5f7d640 1 -- 172.21.15.45:0/497087490 wait complete. 2024-09-10T10:02:08.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:08 smithi162 ceph-mon[34963]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-10T10:02:08.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:08 smithi045 ceph-mon[31114]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-10T10:02:08.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:08 smithi047 ceph-mon[35244]: pgmap v292: 97 pgs: 97 active+clean; 638 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-09-10T10:02:09.777 INFO:teuthology.orchestra.run.smithi045.stdout:34359738443 2024-09-10T10:02:09.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T10:02:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 -- 172.21.15.45:0/1792573827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 msgr2=0x7fc338100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 --2- 172.21.15.45:0/1792573827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 0x7fc338100ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc328009930 tx=0x7fc32802a420 comp rx=0 tx=0).stop 2024-09-10T10:02:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 -- 172.21.15.45:0/1792573827 shutdown_connections 2024-09-10T10:02:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 --2- 172.21.15.45:0/1792573827 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc338101420 0x7fc33810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 --2- 172.21.15.45:0/1792573827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 0x7fc338100ee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 --2- 172.21.15.45:0/1792573827 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc338106a80 0x7fc338106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 -- 172.21.15.45:0/1792573827 >> 172.21.15.45:0/1792573827 conn(0x7fc3380fc7a0 msgr2=0x7fc3380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 -- 172.21.15.45:0/1792573827 shutdown_connections 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.364+0000 7fc33eb17640 1 -- 172.21.15.45:0/1792573827 wait complete. 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 Processor -- start 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 -- start start 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 0x7fc33819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc338101420 0x7fc33819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc338106a80 0x7fc33819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc338110e80 con 0x7fc338100a80 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc338110d00 con 0x7fc338101420 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33eb17640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc338111000 con 0x7fc338106a80 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 0x7fc33819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 0x7fc33819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47666/0 (socket says 172.21.15.45:47666) 2024-09-10T10:02:10.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 -- 172.21.15.45:0/1138690845 learned_addr learned my addr 172.21.15.45:0/1138690845 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc337fff640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc338101420 0x7fc33819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33d0b4640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc338106a80 0x7fc33819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 -- 172.21.15.45:0/1138690845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc338106a80 msgr2=0x7fc33819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc338106a80 0x7fc33819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 -- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc338101420 msgr2=0x7fc33819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc338101420 0x7fc33819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 -- 172.21.15.45:0/1138690845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33819f5f0 con 0x7fc338100a80 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33d0b4640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc338106a80 0x7fc33819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.365+0000 7fc33c8b3640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 0x7fc33819a640 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e9f0 tx=0x7fc32000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.366+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32000cd80 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.366+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc320010910 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.366+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33819f8e0 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.366+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320018990 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.366+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a71c0 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.367+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc33806b2e0 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.368+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc320010ac0 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc335ffb640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc3240a2c10 0x7fc3240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc337fff640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc3240a2c10 0x7fc3240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc3200cfbe0 con 0x7fc338100a80 2024-09-10T10:02:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc335ffb640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] conn(0x7fc3240ac7c0 0x7fc3240aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 --> [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc320010760 con 0x7fc3240ac7c0 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fc3200cff90 con 0x7fc338100a80 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc33d0b4640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] conn(0x7fc3240ac7c0 0x7fc3240aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc337fff640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc3240a2c10 0x7fc3240a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc328009900 tx=0x7fc3280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.369+0000 7fc33d0b4640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] conn(0x7fc3240ac7c0 0x7fc3240aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.370+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== osd.4 v2:172.21.15.47:6808/2470611139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc320010760 con 0x7fc3240ac7c0 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.384+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 --> [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc3381021c0 con 0x7fc3240ac7c0 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc335ffb640 1 -- 172.21.15.45:0/1138690845 <== osd.4 v2:172.21.15.47:6808/2470611139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc3381021c0 con 0x7fc3240ac7c0 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] conn(0x7fc3240ac7c0 msgr2=0x7fc3240aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] conn(0x7fc3240ac7c0 0x7fc3240aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc3240a2c10 msgr2=0x7fc3240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc3240a2c10 0x7fc3240a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc328009900 tx=0x7fc3280058d0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 msgr2=0x7fc33819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 --2- 172.21.15.45:0/1138690845 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc338100a80 0x7fc33819a640 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc32000e9f0 tx=0x7fc32000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:02:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33d0b4640 1 -- 172.21.15.45:0/1138690845 reap_dead start 2024-09-10T10:02:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 shutdown_connections 2024-09-10T10:02:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 >> 172.21.15.45:0/1138690845 conn(0x7fc3380fc7a0 msgr2=0x7fc33806a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 shutdown_connections 2024-09-10T10:02:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:10.385+0000 7fc33eb17640 1 -- 172.21.15.45:0/1138690845 wait complete. 2024-09-10T10:02:10.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:10 smithi162 ceph-mon[34963]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-10T10:02:10.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:10 smithi045 ceph-mon[31114]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-10T10:02:10.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:10 smithi047 ceph-mon[35244]: pgmap v293: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-10T10:02:10.792 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:11.306 INFO:teuthology.orchestra.run.smithi045.stdout:55834574917 2024-09-10T10:02:11.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T10:02:12.159 INFO:teuthology.orchestra.run.smithi045.stdout:137438953527 2024-09-10T10:02:12.160 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T10:02:12.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:12 smithi045 ceph-mon[31114]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-09-10T10:02:12.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:12 smithi162 ceph-mon[34963]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-09-10T10:02:12.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:12 smithi047 ceph-mon[35244]: pgmap v294: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 2 op/s 2024-09-10T10:02:13.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.010+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/580088371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 msgr2=0x7f6a78101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.010+0000 7f6a7f3d3640 1 --2- 172.21.15.45:0/580088371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 0x7f6a78101660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009900 tx=0x7f6a6802a410 comp rx=0 tx=0).stop 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/580088371 shutdown_connections 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 --2- 172.21.15.45:0/580088371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a78101ba0 0x7f6a7810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 --2- 172.21.15.45:0/580088371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 0x7f6a78101660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 --2- 172.21.15.45:0/580088371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a78105ad0 0x7f6a78105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/580088371 >> 172.21.15.45:0/580088371 conn(0x7f6a78077210 msgr2=0x7f6a780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/580088371 shutdown_connections 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/580088371 wait complete. 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.011+0000 7f6a7f3d3640 1 Processor -- start 2024-09-10T10:02:13.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7f3d3640 1 -- start start 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7f3d3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 0x7f6a78199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7f3d3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a78101ba0 0x7f6a781997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7f3d3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a78105ad0 0x7f6a781a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7f3d3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78110ef0 con 0x7f6a78101200 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7f3d3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78110d70 con 0x7f6a78101ba0 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7f3d3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78111070 con 0x7f6a78105ad0 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 0x7f6a78199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 0x7f6a78199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47720/0 (socket says 172.21.15.45:47720) 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 -- 172.21.15.45:0/4112417759 learned_addr learned my addr 172.21.15.45:0/4112417759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:13.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7c96e640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a78101ba0 0x7f6a781997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d970640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a78105ad0 0x7f6a781a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 -- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a78105ad0 msgr2=0x7f6a781a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6a78105ad0 0x7f6a781a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 -- 172.21.15.45:0/4112417759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a78101ba0 msgr2=0x7f6a781997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a78101ba0 0x7f6a781997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7d16f640 1 -- 172.21.15.45:0/4112417759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a781a4820 con 0x7f6a78101200 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.012+0000 7f6a7c96e640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6a78101ba0 0x7f6a781997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:13.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.013+0000 7f6a7d16f640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 0x7f6a78199260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d950 tx=0x7f6a7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.013+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74017030 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.013+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a781a49f0 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.013+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a74004020 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.013+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a740047a0 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.013+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a781a4f00 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.014+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a78104410 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.015+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6a740041d0 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a6e7fc640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a640a2c10 0x7f6a640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a7c96e640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a640a2c10 0x7f6a640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6a740cf210 con 0x7f6a78101200 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a6e7fc640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] conn(0x7f6a640ac7c0 0x7f6a640aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 --> [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a74021e70 con 0x7f6a640ac7c0 2024-09-10T10:02:13.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f6a740cf660 con 0x7f6a78101200 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a7c96e640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a640a2c10 0x7f6a640a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009230 tx=0x7f6a680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.016+0000 7f6a7d970640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] conn(0x7f6a640ac7c0 0x7f6a640aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.017+0000 7f6a7d970640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] conn(0x7f6a640ac7c0 0x7f6a640aec60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.018+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== osd.11 v2:172.21.15.162:6824/1692831932 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a74021e70 con 0x7f6a640ac7c0 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.031+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 --> [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6a78102210 con 0x7f6a640ac7c0 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a6e7fc640 1 -- 172.21.15.45:0/4112417759 <== osd.11 v2:172.21.15.162:6824/1692831932 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6a78102210 con 0x7f6a640ac7c0 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] conn(0x7f6a640ac7c0 msgr2=0x7f6a640aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] conn(0x7f6a640ac7c0 0x7f6a640aec60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a640a2c10 msgr2=0x7f6a640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6a640a2c10 0x7f6a640a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009230 tx=0x7f6a680058d0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 msgr2=0x7f6a78199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 --2- 172.21.15.45:0/4112417759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a78101200 0x7f6a78199260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d950 tx=0x7f6a7400de10 comp rx=0 tx=0).stop 2024-09-10T10:02:13.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7d970640 1 -- 172.21.15.45:0/4112417759 reap_dead start 2024-09-10T10:02:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 shutdown_connections 2024-09-10T10:02:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 >> 172.21.15.45:0/4112417759 conn(0x7f6a78077210 msgr2=0x7f6a781039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 shutdown_connections 2024-09-10T10:02:13.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:13.032+0000 7f6a7f3d3640 1 -- 172.21.15.45:0/4112417759 wait complete. 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 -- 172.21.15.45:0/4240758816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c106a80 msgr2=0x7f6c8c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 --2- 172.21.15.45:0/4240758816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c106a80 0x7f6c8c106e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009930 tx=0x7f6c7c02a420 comp rx=0 tx=0).stop 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 -- 172.21.15.45:0/4240758816 shutdown_connections 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 --2- 172.21.15.45:0/4240758816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c8c101420 0x7f6c8c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 --2- 172.21.15.45:0/4240758816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c8c100a80 0x7f6c8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 --2- 172.21.15.45:0/4240758816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c106a80 0x7f6c8c106e60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 -- 172.21.15.45:0/4240758816 >> 172.21.15.45:0/4240758816 conn(0x7f6c8c0fc7a0 msgr2=0x7f6c8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 -- 172.21.15.45:0/4240758816 shutdown_connections 2024-09-10T10:02:13.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.700+0000 7f6c918cd640 1 -- 172.21.15.45:0/4240758816 wait complete. 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 Processor -- start 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 -- start start 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c8c100a80 0x7f6c8c0fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c101420 0x7f6c8c0fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c8c106a80 0x7f6c8c0fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c110e80 con 0x7f6c8c101420 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c110d00 con 0x7f6c8c100a80 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c918cd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c111000 con 0x7f6c8c106a80 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c101420 0x7f6c8c0fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c101420 0x7f6c8c0fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47688/0 (socket says 172.21.15.45:47688) 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.701+0000 7f6c83fff640 1 -- 172.21.15.45:0/2584387246 learned_addr learned my addr 172.21.15.45:0/2584387246 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c8affd640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c8c100a80 0x7f6c8c0fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c8b7fe640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c8c106a80 0x7f6c8c0fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c83fff640 1 -- 172.21.15.45:0/2584387246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c8c106a80 msgr2=0x7f6c8c0fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c83fff640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6c8c106a80 0x7f6c8c0fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c83fff640 1 -- 172.21.15.45:0/2584387246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c8c100a80 msgr2=0x7f6c8c0fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c83fff640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c8c100a80 0x7f6c8c0fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c83fff640 1 -- 172.21.15.45:0/2584387246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c1a68a0 con 0x7f6c8c101420 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c8affd640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c8c100a80 0x7f6c8c0fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c83fff640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c101420 0x7f6c8c0fd300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400d950 tx=0x7f6c7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c74017030 con 0x7f6c8c101420 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c1a6b90 con 0x7f6c8c101420 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c1a7120 con 0x7f6c8c101420 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c74004020 con 0x7f6c8c101420 2024-09-10T10:02:13.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.702+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c740047a0 con 0x7f6c8c101420 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.703+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c6c001060 con 0x7f6c8c101420 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.704+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6c740041d0 con 0x7f6c8c101420 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.705+0000 7f6c908cb640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6c780a2c10 0x7f6c780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.705+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6c740cef40 con 0x7f6c8c101420 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.705+0000 7f6c8affd640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6c780a2c10 0x7f6c780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.706+0000 7f6c908cb640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] conn(0x7f6c780ac7c0 0x7f6c780aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.706+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 --> [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c780af330 con 0x7f6c780ac7c0 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.706+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f6c740cf390 con 0x7f6c8c101420 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.706+0000 7f6c8b7fe640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] conn(0x7f6c780ac7c0 0x7f6c780aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.706+0000 7f6c8affd640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6c780a2c10 0x7f6c780a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000c00 tx=0x7f6c7c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.706+0000 7f6c8b7fe640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] conn(0x7f6c780ac7c0 0x7f6c780aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:13.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.707+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== osd.3 v2:172.21.15.45:6810/2620350256 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c780af330 con 0x7f6c780ac7c0 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.721+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 --> [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6c6c002e00 con 0x7f6c780ac7c0 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.721+0000 7f6c908cb640 1 -- 172.21.15.45:0/2584387246 <== osd.3 v2:172.21.15.45:6810/2620350256 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6c6c002e00 con 0x7f6c780ac7c0 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] conn(0x7f6c780ac7c0 msgr2=0x7f6c780aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] conn(0x7f6c780ac7c0 0x7f6c780aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6c780a2c10 msgr2=0x7f6c780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f6c780a2c10 0x7f6c780a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000c00 tx=0x7f6c7c005a20 comp rx=0 tx=0).stop 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c101420 msgr2=0x7f6c8c0fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 --2- 172.21.15.45:0/2584387246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c101420 0x7f6c8c0fd300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400d950 tx=0x7f6c7400de10 comp rx=0 tx=0).stop 2024-09-10T10:02:13.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c8b7fe640 1 -- 172.21.15.45:0/2584387246 reap_dead start 2024-09-10T10:02:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 shutdown_connections 2024-09-10T10:02:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 >> 172.21.15.45:0/2584387246 conn(0x7f6c8c0fc7a0 msgr2=0x7f6c8c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 shutdown_connections 2024-09-10T10:02:13.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.722+0000 7f6c918cd640 1 -- 172.21.15.45:0/2584387246 wait complete. 2024-09-10T10:02:14.069 INFO:teuthology.orchestra.run.smithi045.stdout:103079215164 2024-09-10T10:02:14.069 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T10:02:14.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 -- 172.21.15.45:0/3990612221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc101de0 msgr2=0x7fabfc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 --2- 172.21.15.45:0/3990612221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc101de0 0x7fabfc1021c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009900 tx=0x7fabf802a410 comp rx=0 tx=0).stop 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 -- 172.21.15.45:0/3990612221 shutdown_connections 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 --2- 172.21.15.45:0/3990612221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabfc106f10 0x7fabfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 --2- 172.21.15.45:0/3990612221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabfc102790 0x7fabfc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 --2- 172.21.15.45:0/3990612221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc101de0 0x7fabfc1021c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 -- 172.21.15.45:0/3990612221 >> 172.21.15.45:0/3990612221 conn(0x7fabfc0fd8c0 msgr2=0x7fabfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 -- 172.21.15.45:0/3990612221 shutdown_connections 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.900+0000 7fac03778640 1 -- 172.21.15.45:0/3990612221 wait complete. 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 Processor -- start 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 -- start start 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabfc101de0 0x7fabfc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc102790 0x7fabfc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabfc106f10 0x7fabfc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc111f90 con 0x7fabfc102790 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc111e10 con 0x7fabfc101de0 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac03778640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc112110 con 0x7fabfc106f10 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac00d13640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc102790 0x7fabfc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac00d13640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc102790 0x7fabfc198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47704/0 (socket says 172.21.15.45:47704) 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac00d13640 1 -- 172.21.15.45:0/2108229488 learned_addr learned my addr 172.21.15.45:0/2108229488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac01514640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabfc101de0 0x7fabfc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.901+0000 7fac01d15640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabfc106f10 0x7fabfc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac00d13640 1 -- 172.21.15.45:0/2108229488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabfc106f10 msgr2=0x7fabfc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac00d13640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fabfc106f10 0x7fabfc19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac00d13640 1 -- 172.21.15.45:0/2108229488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabfc101de0 msgr2=0x7fabfc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac00d13640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fabfc101de0 0x7fabfc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac00d13640 1 -- 172.21.15.45:0/2108229488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc19d4c0 con 0x7fabfc102790 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac00d13640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc102790 0x7fabfc198a50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b4a0 tx=0x7fabec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec014070 con 0x7fabfc102790 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabec010af0 con 0x7fabfc102790 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc19d7b0 con 0x7fabfc102790 2024-09-10T10:02:14.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec017d10 con 0x7fabfc102790 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.902+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a5090 con 0x7fabfc102790 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.903+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabfc062980 con 0x7fabfc102790 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.904+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fabec017070 con 0x7fabfc102790 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fabf27fc640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fabe80a2c90 0x7fabe80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fac01514640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fabe80a2c90 0x7fabe80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fabec0ce7a0 con 0x7fabfc102790 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fabf27fc640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] conn(0x7fabe80ac840 0x7fabe80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 --> [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabec01ae50 con 0x7fabe80ac840 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fabec0cebf0 con 0x7fabfc102790 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fac01d15640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] conn(0x7fabe80ac840 0x7fabe80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fac01514640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fabe80a2c90 0x7fabe80a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009230 tx=0x7fabf80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.905+0000 7fac01d15640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] conn(0x7fabe80ac840 0x7fabe80aece0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:14.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.906+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== osd.9 v2:172.21.15.45:6826/4106016815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabec01ae50 con 0x7fabe80ac840 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 --> [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabfc062d80 con 0x7fabe80ac840 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fabf27fc640 1 -- 172.21.15.45:0/2108229488 <== osd.9 v2:172.21.15.45:6826/4106016815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fabfc062d80 con 0x7fabe80ac840 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] conn(0x7fabe80ac840 msgr2=0x7fabe80aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] conn(0x7fabe80ac840 0x7fabe80aece0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fabe80a2c90 msgr2=0x7fabe80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fabe80a2c90 0x7fabe80a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009230 tx=0x7fabf80058d0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc102790 msgr2=0x7fabfc198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 --2- 172.21.15.45:0/2108229488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fabfc102790 0x7fabfc198a50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fabec00b4a0 tx=0x7fabec00b960 comp rx=0 tx=0).stop 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac01d15640 1 -- 172.21.15.45:0/2108229488 reap_dead start 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 shutdown_connections 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 >> 172.21.15.45:0/2108229488 conn(0x7fabfc0fd8c0 msgr2=0x7fabfc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 shutdown_connections 2024-09-10T10:02:14.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:12.920+0000 7fac03778640 1 -- 172.21.15.45:0/2108229488 wait complete. 2024-09-10T10:02:14.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:14 smithi162 ceph-mon[34963]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 2 op/s 2024-09-10T10:02:14.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:14 smithi045 ceph-mon[31114]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 2 op/s 2024-09-10T10:02:14.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:14 smithi047 ceph-mon[35244]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 938 B/s wr, 2 op/s 2024-09-10T10:02:14.822 INFO:teuthology.orchestra.run.smithi045.stdout:240518168610 2024-09-10T10:02:14.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.984+0000 7fa06f3da640 1 -- 172.21.15.45:0/14573264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 msgr2=0x7fa0681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.984+0000 7fa06f3da640 1 --2- 172.21.15.45:0/14573264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 0x7fa0681029b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa058009900 tx=0x7fa05802a480 comp rx=0 tx=0).stop 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.984+0000 7fa06f3da640 1 -- 172.21.15.45:0/14573264 shutdown_connections 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.984+0000 7fa06f3da640 1 --2- 172.21.15.45:0/14573264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068102ef0 0x7fa06810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.984+0000 7fa06f3da640 1 --2- 172.21.15.45:0/14573264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 0x7fa0681029b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.984+0000 7fa06f3da640 1 --2- 172.21.15.45:0/14573264 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa068100e20 0x7fa068101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.984+0000 7fa06f3da640 1 -- 172.21.15.45:0/14573264 >> 172.21.15.45:0/14573264 conn(0x7fa0680fc7d0 msgr2=0x7fa0680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 -- 172.21.15.45:0/14573264 shutdown_connections 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 -- 172.21.15.45:0/14573264 wait complete. 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 Processor -- start 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 -- start start 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa068100e20 0x7fa068199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 0x7fa0681998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068102ef0 0x7fa0681a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa068110fa0 con 0x7fa068102570 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa068110e20 con 0x7fa068102ef0 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.985+0000 7fa06f3da640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa068111120 con 0x7fa068100e20 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 0x7fa0681998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 0x7fa0681998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52354/0 (socket says 172.21.15.45:52354) 2024-09-10T10:02:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 -- 172.21.15.45:0/610100236 learned_addr learned my addr 172.21.15.45:0/610100236 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06d176640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa068100e20 0x7fa068199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06d977640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068102ef0 0x7fa0681a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 -- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa068100e20 msgr2=0x7fa068199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa068100e20 0x7fa068199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 -- 172.21.15.45:0/610100236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068102ef0 msgr2=0x7fa0681a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068102ef0 0x7fa0681a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 -- 172.21.15.45:0/610100236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0681a4950 con 0x7fa068102570 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06d176640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa068100e20 0x7fa068199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06d977640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa068102ef0 0x7fa0681a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06c975640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 0x7fa0681998d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa058006010 tx=0x7fa058003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05803f070 con 0x7fa068102570 2024-09-10T10:02:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa058031e30 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0681a4b20 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058035030 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.986+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0681a4fd0 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.987+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa068062980 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.989+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa058042070 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.989+0000 7fa05e7fc640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa0540a2c10 0x7fa0540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.989+0000 7fa06d176640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa0540a2c10 0x7fa0540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.989+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa0580f01f0 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.989+0000 7fa05e7fc640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] conn(0x7fa0540ac7c0 0x7fa0540aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.989+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 --> [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa058038e90 con 0x7fa0540ac7c0 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.989+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa0580f0640 con 0x7fa068102570 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.990+0000 7fa06d176640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa0540a2c10 0x7fa0540a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa064009930 tx=0x7fa064008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.990+0000 7fa06d977640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] conn(0x7fa0540ac7c0 0x7fa0540aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.990+0000 7fa06d977640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] conn(0x7fa0540ac7c0 0x7fa0540aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:14.991+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== osd.2 v2:172.21.15.162:6800/572103448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa058038e90 con 0x7fa0540ac7c0 2024-09-10T10:02:15.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.006+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 --> [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa068062e00 con 0x7fa0540ac7c0 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa05e7fc640 1 -- 172.21.15.45:0/610100236 <== osd.2 v2:172.21.15.162:6800/572103448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa068062e00 con 0x7fa0540ac7c0 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] conn(0x7fa0540ac7c0 msgr2=0x7fa0540aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] conn(0x7fa0540ac7c0 0x7fa0540aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa0540a2c10 msgr2=0x7fa0540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa0540a2c10 0x7fa0540a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa064009930 tx=0x7fa064008040 comp rx=0 tx=0).stop 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 msgr2=0x7fa0681998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 --2- 172.21.15.45:0/610100236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068102570 0x7fa0681998d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa058006010 tx=0x7fa058003c80 comp rx=0 tx=0).stop 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06d977640 1 -- 172.21.15.45:0/610100236 reap_dead start 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 shutdown_connections 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 >> 172.21.15.45:0/610100236 conn(0x7fa0680fc7d0 msgr2=0x7fa0680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 shutdown_connections 2024-09-10T10:02:15.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.007+0000 7fa06f3da640 1 -- 172.21.15.45:0/610100236 wait complete. 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 -- 172.21.15.45:0/4236809364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490101de0 msgr2=0x7fa4901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 --2- 172.21.15.45:0/4236809364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490101de0 0x7fa4901021c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c02a410 comp rx=0 tx=0).stop 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 -- 172.21.15.45:0/4236809364 shutdown_connections 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 --2- 172.21.15.45:0/4236809364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa490106f10 0x7fa49010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 --2- 172.21.15.45:0/4236809364 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa490102790 0x7fa4901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 --2- 172.21.15.45:0/4236809364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490101de0 0x7fa4901021c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 -- 172.21.15.45:0/4236809364 >> 172.21.15.45:0/4236809364 conn(0x7fa4900fd8c0 msgr2=0x7fa4900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 -- 172.21.15.45:0/4236809364 shutdown_connections 2024-09-10T10:02:15.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.300+0000 7fa4980e1640 1 -- 172.21.15.45:0/4236809364 wait complete. 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 Processor -- start 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 -- start start 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa490101de0 0x7fa490198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa490102790 0x7fa4901989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490106f10 0x7fa49019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa490111f70 con 0x7fa490106f10 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa490111df0 con 0x7fa490101de0 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa4980e1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901120f0 con 0x7fa490102790 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa49667e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490106f10 0x7fa49019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa49667e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490106f10 0x7fa49019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52370/0 (socket says 172.21.15.45:52370) 2024-09-10T10:02:15.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa49667e640 1 -- 172.21.15.45:0/2510882403 learned_addr learned my addr 172.21.15.45:0/2510882403 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.301+0000 7fa495e7d640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa490101de0 0x7fa490198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa49567c640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa490102790 0x7fa4901989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa49667e640 1 -- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa490102790 msgr2=0x7fa4901989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa49667e640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa490102790 0x7fa4901989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa49667e640 1 -- 172.21.15.45:0/2510882403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa490101de0 msgr2=0x7fa490198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa49667e640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa490101de0 0x7fa490198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa49667e640 1 -- 172.21.15.45:0/2510882403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49019d410 con 0x7fa490106f10 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa495e7d640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa490101de0 0x7fa490198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa49667e640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490106f10 0x7fa49019cd30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cad0 tx=0x7fa48800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48800ae20 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa488005020 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49019d700 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48800f6f0 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.302+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4901a4fe0 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.303+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa490062940 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.304+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa4880051d0 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.305+0000 7fa486ffd640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa47c0a2c90 0x7fa47c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.305+0000 7fa495e7d640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa47c0a2c90 0x7fa47c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.305+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa4880cf5e0 con 0x7fa490106f10 2024-09-10T10:02:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.305+0000 7fa486ffd640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] conn(0x7fa47c0ac840 0x7fa47c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.305+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 --> [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa47c0af3b0 con 0x7fa47c0ac840 2024-09-10T10:02:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.305+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa4880cfa30 con 0x7fa490106f10 2024-09-10T10:02:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.305+0000 7fa495e7d640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa47c0a2c90 0x7fa47c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005ab0 tx=0x7fa48c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.306+0000 7fa49567c640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] conn(0x7fa47c0ac840 0x7fa47c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.306+0000 7fa49567c640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] conn(0x7fa47c0ac840 0x7fa47c0aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.307+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== osd.5 v2:172.21.15.162:6808/2110302929 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa47c0af3b0 con 0x7fa47c0ac840 2024-09-10T10:02:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.320+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 --> [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa490062dc0 con 0x7fa47c0ac840 2024-09-10T10:02:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa486ffd640 1 -- 172.21.15.45:0/2510882403 <== osd.5 v2:172.21.15.162:6808/2110302929 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa490062dc0 con 0x7fa47c0ac840 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] conn(0x7fa47c0ac840 msgr2=0x7fa47c0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa4980e1640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] conn(0x7fa47c0ac840 0x7fa47c0aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa47c0a2c90 msgr2=0x7fa47c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa4980e1640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fa47c0a2c90 0x7fa47c0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005ab0 tx=0x7fa48c005a20 comp rx=0 tx=0).stop 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490106f10 msgr2=0x7fa49019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa4980e1640 1 --2- 172.21.15.45:0/2510882403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa490106f10 0x7fa49019cd30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cad0 tx=0x7fa48800cf90 comp rx=0 tx=0).stop 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.321+0000 7fa49667e640 1 -- 172.21.15.45:0/2510882403 reap_dead start 2024-09-10T10:02:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.322+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 shutdown_connections 2024-09-10T10:02:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.322+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 >> 172.21.15.45:0/2510882403 conn(0x7fa4900fd8c0 msgr2=0x7fa4900fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.322+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 shutdown_connections 2024-09-10T10:02:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:15.322+0000 7fa4980e1640 1 -- 172.21.15.45:0/2510882403 wait complete. 2024-09-10T10:02:15.333 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:15.334 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 -- 172.21.15.45:0/2337823237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c102dd0 msgr2=0x7f964c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 --2- 172.21.15.45:0/2337823237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c102dd0 0x7f964c1031b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9640009930 tx=0x7f964002a420 comp rx=0 tx=0).stop 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 -- 172.21.15.45:0/2337823237 shutdown_connections 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 --2- 172.21.15.45:0/2337823237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f964c108300 0x7f964c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 --2- 172.21.15.45:0/2337823237 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f964c103780 0x7f964c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 --2- 172.21.15.45:0/2337823237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c102dd0 0x7f964c1031b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 -- 172.21.15.45:0/2337823237 >> 172.21.15.45:0/2337823237 conn(0x7f964c0fc7d0 msgr2=0x7f964c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 -- 172.21.15.45:0/2337823237 shutdown_connections 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.370+0000 7f9652f75640 1 -- 172.21.15.45:0/2337823237 wait complete. 2024-09-10T10:02:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 Processor -- start 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 -- start start 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f964c102dd0 0x7f964c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f964c103780 0x7f964c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c108300 0x7f964c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f964c1120e0 con 0x7f964c108300 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f964c111f60 con 0x7f964c102dd0 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9652f75640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f964c112260 con 0x7f964c103780 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9651512640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c108300 0x7f964c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9651512640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c108300 0x7f964c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52388/0 (socket says 172.21.15.45:52388) 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9651512640 1 -- 172.21.15.45:0/3687680147 learned_addr learned my addr 172.21.15.45:0/3687680147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f9650d11640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f964c102dd0 0x7f964c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.371+0000 7f964bfff640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f964c103780 0x7f964c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9651512640 1 -- 172.21.15.45:0/3687680147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f964c103780 msgr2=0x7f964c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9651512640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f964c103780 0x7f964c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9651512640 1 -- 172.21.15.45:0/3687680147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f964c102dd0 msgr2=0x7f964c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9651512640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f964c102dd0 0x7f964c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9651512640 1 -- 172.21.15.45:0/3687680147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c19d570 con 0x7f964c108300 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f964bfff640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f964c103780 0x7f964c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9650d11640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f964c102dd0 0x7f964c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9651512640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c108300 0x7f964c19ce90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f964400eaa0 tx=0x7f964400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400cdb0 con 0x7f964c108300 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9644005020 con 0x7f964c108300 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c19d860 con 0x7f964c108300 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96440116b0 con 0x7f964c108300 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.372+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c1a5140 con 0x7f964c108300 2024-09-10T10:02:16.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.373+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f964c062980 con 0x7f964c108300 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.374+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9644005560 con 0x7f964c108300 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.375+0000 7f9649ffb640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f963c0a2c90 0x7f963c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.375+0000 7f9650d11640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f963c0a2c90 0x7f963c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.375+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f96440cf7a0 con 0x7f964c108300 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.375+0000 7f9649ffb640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] conn(0x7f963c0ac840 0x7f963c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.375+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 --> [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9644014ec0 con 0x7f963c0ac840 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.376+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f96440cfbf0 con 0x7f964c108300 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.376+0000 7f964bfff640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] conn(0x7f963c0ac840 0x7f963c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.376+0000 7f9650d11640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f963c0a2c90 0x7f963c0a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9640006010 tx=0x7f9640005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.376+0000 7f964bfff640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] conn(0x7f963c0ac840 0x7f963c0aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.377+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== osd.6 v2:172.21.15.45:6818/135941807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9644014ec0 con 0x7f963c0ac840 2024-09-10T10:02:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.392+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 --> [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f964c062e00 con 0x7f963c0ac840 2024-09-10T10:02:16.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.392+0000 7f9649ffb640 1 -- 172.21.15.45:0/3687680147 <== osd.6 v2:172.21.15.45:6818/135941807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f964c062e00 con 0x7f963c0ac840 2024-09-10T10:02:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] conn(0x7f963c0ac840 msgr2=0x7f963c0aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] conn(0x7f963c0ac840 0x7f963c0aece0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f963c0a2c90 msgr2=0x7f963c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f963c0a2c90 0x7f963c0a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9640006010 tx=0x7f9640005a20 comp rx=0 tx=0).stop 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c108300 msgr2=0x7f964c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 --2- 172.21.15.45:0/3687680147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f964c108300 0x7f964c19ce90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f964400eaa0 tx=0x7f964400ef60 comp rx=0 tx=0).stop 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9651512640 1 -- 172.21.15.45:0/3687680147 reap_dead start 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 shutdown_connections 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 >> 172.21.15.45:0/3687680147 conn(0x7f964c0fc7d0 msgr2=0x7f964c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.393+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 shutdown_connections 2024-09-10T10:02:16.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:16.394+0000 7f9652f75640 1 -- 172.21.15.45:0/3687680147 wait complete. 2024-09-10T10:02:16.493 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:16 smithi045 ceph-mon[31114]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:02:16.557 INFO:teuthology.orchestra.run.smithi045.stdout:322122547221 2024-09-10T10:02:16.558 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T10:02:16.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:16 smithi162 ceph-mon[34963]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:02:16.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:16 smithi047 ceph-mon[35244]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:02:17.693 INFO:teuthology.orchestra.run.smithi045.stdout:77309411392 2024-09-10T10:02:17.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T10:02:18.146 INFO:teuthology.orchestra.run.smithi045.stdout:158913790002 2024-09-10T10:02:18.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T10:02:18.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.278+0000 7fd4361e8640 1 -- 172.21.15.45:0/4100695030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 msgr2=0x7fd43010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.278+0000 7fd4361e8640 1 --2- 172.21.15.45:0/4100695030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 0x7fd43010e620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd428009ca0 tx=0x7fd42802a950 comp rx=0 tx=0).stop 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 -- 172.21.15.45:0/4100695030 shutdown_connections 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 --2- 172.21.15.45:0/4100695030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 0x7fd43010e620 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 --2- 172.21.15.45:0/4100695030 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430103670 0x7fd43010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 --2- 172.21.15.45:0/4100695030 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd430102d50 0x7fd430103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 -- 172.21.15.45:0/4100695030 >> 172.21.15.45:0/4100695030 conn(0x7fd4300fc7d0 msgr2=0x7fd4300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 -- 172.21.15.45:0/4100695030 shutdown_connections 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 -- 172.21.15.45:0/4100695030 wait complete. 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.279+0000 7fd4361e8640 1 Processor -- start 2024-09-10T10:02:18.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd4361e8640 1 -- start start 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd4361e8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430102d50 0x7fd430198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd4361e8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd430103670 0x7fd430198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd4361e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 0x7fd43019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd4361e8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd430111f40 con 0x7fd43010c180 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd4361e8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd430111dc0 con 0x7fd430103670 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd4361e8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301120c0 con 0x7fd430102d50 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 0x7fd43019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd430103670 0x7fd430198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 0x7fd43019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52408/0 (socket says 172.21.15.45:52408) 2024-09-10T10:02:18.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42effd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd430103670 0x7fd430198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:35148/0 (socket says 172.21.15.45:35148) 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42ffff640 1 -- 172.21.15.45:0/594930905 learned_addr learned my addr 172.21.15.45:0/594930905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42f7fe640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430102d50 0x7fd430198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42effd640 1 -- 172.21.15.45:0/594930905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430102d50 msgr2=0x7fd430198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42effd640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd430102d50 0x7fd430198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42effd640 1 -- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 msgr2=0x7fd43019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42effd640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 0x7fd43019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42effd640 1 -- 172.21.15.45:0/594930905 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43019d3d0 con 0x7fd430103670 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.280+0000 7fd42ffff640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd43010c180 0x7fd43019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.281+0000 7fd42effd640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd430103670 0x7fd430198960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c720 tx=0x7fd42400cbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.281+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424007bb0 con 0x7fd430103670 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.281+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43019d6c0 con 0x7fd430103670 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.281+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42400ce30 con 0x7fd430103670 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.281+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4301008e0 con 0x7fd430103670 2024-09-10T10:02:18.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.281+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424004db0 con 0x7fd430103670 2024-09-10T10:02:18.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.282+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd41c001030 con 0x7fd430103670 2024-09-10T10:02:18.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.283+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd424005000 con 0x7fd430103670 2024-09-10T10:02:18.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.284+0000 7fd42cff9640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd4180a2c10 0x7fd4180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.284+0000 7fd42f7fe640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd4180a2c10 0x7fd4180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.284+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd4240cef40 con 0x7fd430103670 2024-09-10T10:02:18.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.285+0000 7fd42cff9640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] conn(0x7fd4180ac7c0 0x7fd4180aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:18.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.285+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 --> [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd424012ec0 con 0x7fd4180ac7c0 2024-09-10T10:02:18.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.285+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd4240cf2f0 con 0x7fd430103670 2024-09-10T10:02:18.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.285+0000 7fd42f7fe640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd4180a2c10 0x7fd4180a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd4200096c0 tx=0x7fd420016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.285+0000 7fd42ffff640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] conn(0x7fd4180ac7c0 0x7fd4180aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:18.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.285+0000 7fd42ffff640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] conn(0x7fd4180ac7c0 0x7fd4180aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:18.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.286+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== osd.10 v2:172.21.15.47:6824/3408198504 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd424012ec0 con 0x7fd4180ac7c0 2024-09-10T10:02:18.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.304+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 --> [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd41c002dd0 con 0x7fd4180ac7c0 2024-09-10T10:02:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.305+0000 7fd42cff9640 1 -- 172.21.15.45:0/594930905 <== osd.10 v2:172.21.15.47:6824/3408198504 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd41c002dd0 con 0x7fd4180ac7c0 2024-09-10T10:02:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.305+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] conn(0x7fd4180ac7c0 msgr2=0x7fd4180aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.305+0000 7fd4361e8640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] conn(0x7fd4180ac7c0 0x7fd4180aec60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.305+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd4180a2c10 msgr2=0x7fd4180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.305+0000 7fd4361e8640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd4180a2c10 0x7fd4180a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd4200096c0 tx=0x7fd420016040 comp rx=0 tx=0).stop 2024-09-10T10:02:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.306+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd430103670 msgr2=0x7fd430198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.306+0000 7fd4361e8640 1 --2- 172.21.15.45:0/594930905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd430103670 0x7fd430198960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c720 tx=0x7fd42400cbe0 comp rx=0 tx=0).stop 2024-09-10T10:02:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.306+0000 7fd42ffff640 1 -- 172.21.15.45:0/594930905 reap_dead start 2024-09-10T10:02:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.306+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 shutdown_connections 2024-09-10T10:02:18.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.306+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 >> 172.21.15.45:0/594930905 conn(0x7fd4300fc7d0 msgr2=0x7fd4300fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.306+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 shutdown_connections 2024-09-10T10:02:18.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:18.306+0000 7fd4361e8640 1 -- 172.21.15.45:0/594930905 wait complete. 2024-09-10T10:02:18.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:18 smithi162 ceph-mon[34963]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:02:18.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:18 smithi045 ceph-mon[31114]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:02:18.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:18 smithi047 ceph-mon[35244]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:02:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 -- 172.21.15.45:0/1927080130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 msgr2=0x7fea14107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 --2- 172.21.15.45:0/1927080130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 0x7fea14107db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fea04009cd0 tx=0x7fea0402a8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 -- 172.21.15.45:0/1927080130 shutdown_connections 2024-09-10T10:02:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 --2- 172.21.15.45:0/1927080130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea14102370 0x7fea1410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 --2- 172.21.15.45:0/1927080130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea141019d0 0x7fea14101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 --2- 172.21.15.45:0/1927080130 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 0x7fea14107db0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 -- 172.21.15.45:0/1927080130 >> 172.21.15.45:0/1927080130 conn(0x7fea140fd710 msgr2=0x7fea140ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 -- 172.21.15.45:0/1927080130 shutdown_connections 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.397+0000 7fea1af9e640 1 -- 172.21.15.45:0/1927080130 wait complete. 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 Processor -- start 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 -- start start 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea141019d0 0x7fea141982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea14102370 0x7fea14198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 0x7fea1419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111dd0 con 0x7fea141079d0 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111c50 con 0x7fea14102370 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1af9e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fea14111f50 con 0x7fea141019d0 2024-09-10T10:02:19.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1a79d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 0x7fea1419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1a79d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 0x7fea1419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52434/0 (socket says 172.21.15.45:52434) 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1a79d640 1 -- 172.21.15.45:0/4226770300 learned_addr learned my addr 172.21.15.45:0/4226770300 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea1979b640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea14102370 0x7fea14198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.398+0000 7fea19f9c640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea141019d0 0x7fea141982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1a79d640 1 -- 172.21.15.45:0/4226770300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea141019d0 msgr2=0x7fea141982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1a79d640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea141019d0 0x7fea141982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1a79d640 1 -- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea14102370 msgr2=0x7fea14198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1a79d640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea14102370 0x7fea14198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1a79d640 1 -- 172.21.15.45:0/4226770300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1419d280 con 0x7fea141079d0 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1a79d640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 0x7fea1419cba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c970 tx=0x7fea0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c007bb0 con 0x7fea141079d0 2024-09-10T10:02:19.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1419d570 con 0x7fea141079d0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea0c007d50 con 0x7fea141079d0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00f450 con 0x7fea141079d0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.399+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea141a4e50 con 0x7fea141079d0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.400+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea14062980 con 0x7fea141079d0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.401+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea0c004f00 con 0x7fea141079d0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.402+0000 7fea12ffd640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe9fc0a2c10 0x7fe9fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.402+0000 7fea19f9c640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe9fc0a2c10 0x7fe9fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.402+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fea0c0cf320 con 0x7fea141079d0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.402+0000 7fea12ffd640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] conn(0x7fe9fc0ac7c0 0x7fe9fc0aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.402+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 --> [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea0c012ec0 con 0x7fe9fc0ac7c0 2024-09-10T10:02:19.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.402+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fea0c0cf6d0 con 0x7fea141079d0 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.403+0000 7fea19f9c640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe9fc0a2c10 0x7fe9fc0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea04006010 tx=0x7fea040093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.403+0000 7fea1979b640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] conn(0x7fe9fc0ac7c0 0x7fe9fc0aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.403+0000 7fea1979b640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] conn(0x7fe9fc0ac7c0 0x7fe9fc0aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.404+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== osd.7 v2:172.21.15.47:6816/2101068416 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea0c012ec0 con 0x7fe9fc0ac7c0 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.418+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 --> [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea14062d80 con 0x7fe9fc0ac7c0 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.418+0000 7fea12ffd640 1 -- 172.21.15.45:0/4226770300 <== osd.7 v2:172.21.15.47:6816/2101068416 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea14062d80 con 0x7fe9fc0ac7c0 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] conn(0x7fe9fc0ac7c0 msgr2=0x7fe9fc0aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] conn(0x7fe9fc0ac7c0 0x7fe9fc0aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe9fc0a2c10 msgr2=0x7fe9fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe9fc0a2c10 0x7fe9fc0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea04006010 tx=0x7fea040093f0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 msgr2=0x7fea1419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 --2- 172.21.15.45:0/4226770300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea141079d0 0x7fea1419cba0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00c970 tx=0x7fea0c00ce30 comp rx=0 tx=0).stop 2024-09-10T10:02:19.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1a79d640 1 -- 172.21.15.45:0/4226770300 reap_dead start 2024-09-10T10:02:19.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 shutdown_connections 2024-09-10T10:02:19.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 >> 172.21.15.45:0/4226770300 conn(0x7fea140fd710 msgr2=0x7fea140fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 shutdown_connections 2024-09-10T10:02:19.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:19.419+0000 7fea1af9e640 1 -- 172.21.15.45:0/4226770300 wait complete. 2024-09-10T10:02:19.430 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:19.431 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:19.431 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:19.814 INFO:teuthology.orchestra.run.smithi045.stdout:270582939678 2024-09-10T10:02:19.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T10:02:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 -- 172.21.15.45:0/3757375631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4103760 msgr2=0x7fd4a4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/3757375631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4103760 0x7fd4a4107ba0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd498009930 tx=0x7fd49802a420 comp rx=0 tx=0).stop 2024-09-10T10:02:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 -- 172.21.15.45:0/3757375631 shutdown_connections 2024-09-10T10:02:20.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/3757375631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a41082e0 0x7fd4a410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/3757375631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4103760 0x7fd4a4107ba0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/3757375631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a4102db0 0x7fd4a4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 -- 172.21.15.45:0/3757375631 >> 172.21.15.45:0/3757375631 conn(0x7fd4a40fc7d0 msgr2=0x7fd4a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 -- 172.21.15.45:0/3757375631 shutdown_connections 2024-09-10T10:02:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 -- 172.21.15.45:0/3757375631 wait complete. 2024-09-10T10:02:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 Processor -- start 2024-09-10T10:02:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.194+0000 7fd4aaae1640 1 -- start start 2024-09-10T10:02:20.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4aaae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4102db0 0x7fd4a4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4aaae1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a4103760 0x7fd4a4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4aaae1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a41082e0 0x7fd4a419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4aaae1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4112050 con 0x7fd4a4102db0 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4aaae1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4111ed0 con 0x7fd4a4103760 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4aaae1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a41121d0 con 0x7fd4a41082e0 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4102db0 0x7fd4a4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4102db0 0x7fd4a4198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52448/0 (socket says 172.21.15.45:52448) 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 -- 172.21.15.45:0/2153293263 learned_addr learned my addr 172.21.15.45:0/2153293263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a907e640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a41082e0 0x7fd4a419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a3fff640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a4103760 0x7fd4a4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 -- 172.21.15.45:0/2153293263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a41082e0 msgr2=0x7fd4a419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a41082e0 0x7fd4a419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 -- 172.21.15.45:0/2153293263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a4103760 msgr2=0x7fd4a4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a4103760 0x7fd4a4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 -- 172.21.15.45:0/2153293263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a419d500 con 0x7fd4a4102db0 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a907e640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a41082e0 0x7fd4a419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.195+0000 7fd4a887d640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4102db0 0x7fd4a4198550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd49400d950 tx=0x7fd49400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.196+0000 7fd4a1ffb640 1 -- 172.21.15.45:0/2153293263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494017070 con 0x7fd4a4102db0 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.196+0000 7fd4a1ffb640 1 -- 172.21.15.45:0/2153293263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd494009c30 con 0x7fd4a4102db0 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.196+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a419d7f0 con 0x7fd4a4102db0 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.196+0000 7fd4a1ffb640 1 -- 172.21.15.45:0/2153293263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400f6b0 con 0x7fd4a4102db0 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.196+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a41a50d0 con 0x7fd4a4102db0 2024-09-10T10:02:20.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.197+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a4103190 con 0x7fd4a4102db0 2024-09-10T10:02:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.198+0000 7fd4a1ffb640 1 -- 172.21.15.45:0/2153293263 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd4940109d0 con 0x7fd4a4102db0 2024-09-10T10:02:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.199+0000 7fd4a1ffb640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd48c0a2c90 0x7fd48c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.199+0000 7fd4a3fff640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd48c0a2c90 0x7fd48c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.199+0000 7fd4a1ffb640 1 -- 172.21.15.45:0/2153293263 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd4940ceec0 con 0x7fd4a4102db0 2024-09-10T10:02:20.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.199+0000 7fd4a3fff640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd48c0a2c90 0x7fd48c0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd498009900 tx=0x7fd498005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.203+0000 7fd4a1ffb640 1 -- 172.21.15.45:0/2153293263 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd494009de0 con 0x7fd4a4102db0 2024-09-10T10:02:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.383+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd4a410a2c0 con 0x7fd4a4102db0 2024-09-10T10:02:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.383+0000 7fd4a1ffb640 1 -- 172.21.15.45:0/2153293263 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4940bb070 con 0x7fd4a4102db0 2024-09-10T10:02:20.385 INFO:teuthology.orchestra.run.smithi045.stdout:34359738445 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.385+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd48c0a2c90 msgr2=0x7fd48c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.385+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd48c0a2c90 0x7fd48c0a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd498009900 tx=0x7fd498005c00 comp rx=0 tx=0).stop 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.385+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4102db0 msgr2=0x7fd4a4198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.385+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4102db0 0x7fd4a4198550 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd49400d950 tx=0x7fd49400de10 comp rx=0 tx=0).stop 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 shutdown_connections 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd48c0a2c90 0x7fd48c0a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd4a41082e0 0x7fd4a419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4a4103760 0x7fd4a4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 --2- 172.21.15.45:0/2153293263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4a4102db0 0x7fd4a4198550 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 >> 172.21.15.45:0/2153293263 conn(0x7fd4a40fc7d0 msgr2=0x7fd4a4106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 shutdown_connections 2024-09-10T10:02:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:20.386+0000 7fd4aaae1640 1 -- 172.21.15.45:0/2153293263 wait complete. 2024-09-10T10:02:20.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:20 smithi162 ceph-mon[34963]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:02:20.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:20 smithi045 ceph-mon[31114]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:02:20.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:20 smithi047 ceph-mon[35244]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:02:20.713 INFO:teuthology.orchestra.run.smithi045.stdout:188978561070 2024-09-10T10:02:20.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T10:02:21.349 INFO:teuthology.orchestra.run.smithi045.stdout:210453397546 2024-09-10T10:02:21.349 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T10:02:21.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:21 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2153293263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:02:21.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:21 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2153293263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:02:21.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:21 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2153293263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T10:02:21.692 INFO:teuthology.orchestra.run.smithi045.stdout:292057776154 2024-09-10T10:02:21.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T10:02:22.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.132+0000 7fe865832640 1 -- 172.21.15.45:0/633365191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860106a70 msgr2=0x7fe860106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.132+0000 7fe865832640 1 --2- 172.21.15.45:0/633365191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860106a70 0x7fe860106e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe850009cd0 tx=0x7fe85002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 -- 172.21.15.45:0/633365191 shutdown_connections 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 --2- 172.21.15.45:0/633365191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe860101410 0x7fe86010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 --2- 172.21.15.45:0/633365191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe860100a70 0x7fe860100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 --2- 172.21.15.45:0/633365191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860106a70 0x7fe860106e50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 -- 172.21.15.45:0/633365191 >> 172.21.15.45:0/633365191 conn(0x7fe8600fc790 msgr2=0x7fe8600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 -- 172.21.15.45:0/633365191 shutdown_connections 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 -- 172.21.15.45:0/633365191 wait complete. 2024-09-10T10:02:22.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 Processor -- start 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.133+0000 7fe865832640 1 -- start start 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe865832640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860100a70 0x7fe8601984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe865832640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe860101410 0x7fe8601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe865832640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe860106a70 0x7fe86019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe865832640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe860110e30 con 0x7fe860100a70 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe865832640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe860110cb0 con 0x7fe860101410 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe865832640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe860110fb0 con 0x7fe860106a70 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860100a70 0x7fe8601984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860100a70 0x7fe8601984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52464/0 (socket says 172.21.15.45:52464) 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 -- 172.21.15.45:0/1361932518 learned_addr learned my addr 172.21.15.45:0/1361932518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85e7fc640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe860101410 0x7fe8601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85f7fe640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe860106a70 0x7fe86019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 -- 172.21.15.45:0/1361932518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe860106a70 msgr2=0x7fe86019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe860106a70 0x7fe86019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 -- 172.21.15.45:0/1361932518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe860101410 msgr2=0x7fe8601989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe860101410 0x7fe8601989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 -- 172.21.15.45:0/1361932518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86019d460 con 0x7fe860100a70 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85e7fc640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe860101410 0x7fe8601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.134+0000 7fe85effd640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860100a70 0x7fe8601984b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe85002ae00 tx=0x7fe850003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.135+0000 7fe864830640 1 -- 172.21.15.45:0/1361932518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850004040 con 0x7fe860100a70 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.135+0000 7fe864830640 1 -- 172.21.15.45:0/1361932518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8500046c0 con 0x7fe860100a70 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.135+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86019d6f0 con 0x7fe860100a70 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.135+0000 7fe864830640 1 -- 172.21.15.45:0/1361932518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850046480 con 0x7fe860100a70 2024-09-10T10:02:22.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.135+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8601a5030 con 0x7fe860100a70 2024-09-10T10:02:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.136+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe860100ed0 con 0x7fe860100a70 2024-09-10T10:02:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.137+0000 7fe864830640 1 -- 172.21.15.45:0/1361932518 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8500305a0 con 0x7fe860100a70 2024-09-10T10:02:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.138+0000 7fe864830640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe84c0a2c10 0x7fe84c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.138+0000 7fe85e7fc640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe84c0a2c10 0x7fe84c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.138+0000 7fe864830640 1 -- 172.21.15.45:0/1361932518 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe85002e030 con 0x7fe860100a70 2024-09-10T10:02:22.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.139+0000 7fe85e7fc640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe84c0a2c10 0x7fe84c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe8601052a0 tx=0x7fe84800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.142+0000 7fe864830640 1 -- 172.21.15.45:0/1361932518 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe85004b060 con 0x7fe860100a70 2024-09-10T10:02:22.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.323+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe860109ff0 con 0x7fe860100a70 2024-09-10T10:02:22.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.324+0000 7fe864830640 1 -- 172.21.15.45:0/1361932518 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe850030890 con 0x7fe860100a70 2024-09-10T10:02:22.325 INFO:teuthology.orchestra.run.smithi045.stdout:55834574920 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.325+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe84c0a2c10 msgr2=0x7fe84c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.325+0000 7fe865832640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe84c0a2c10 0x7fe84c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe8601052a0 tx=0x7fe84800b390 comp rx=0 tx=0).stop 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.325+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860100a70 msgr2=0x7fe8601984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.325+0000 7fe865832640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860100a70 0x7fe8601984b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe85002ae00 tx=0x7fe850003970 comp rx=0 tx=0).stop 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 shutdown_connections 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe84c0a2c10 0x7fe84c0a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe860106a70 0x7fe86019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe860101410 0x7fe8601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 --2- 172.21.15.45:0/1361932518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860100a70 0x7fe8601984b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 >> 172.21.15.45:0/1361932518 conn(0x7fe8600fc790 msgr2=0x7fe86006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 shutdown_connections 2024-09-10T10:02:22.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.326+0000 7fe865832640 1 -- 172.21.15.45:0/1361932518 wait complete. 2024-09-10T10:02:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:22 smithi162 ceph-mon[34963]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-10T10:02:22.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:22 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1361932518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:02:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:22 smithi045 ceph-mon[31114]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-10T10:02:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:22 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1361932518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:02:22.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:22 smithi047 ceph-mon[35244]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-10T10:02:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:22 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1361932518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T10:02:22.869 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738445 for osd.0 2024-09-10T10:02:22.869 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 -- 172.21.15.45:0/2030064871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 msgr2=0x7f7958100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 --2- 172.21.15.45:0/2030064871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f7958100ee0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f794c009930 tx=0x7f794c02a420 comp rx=0 tx=0).stop 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 -- 172.21.15.45:0/2030064871 shutdown_connections 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 --2- 172.21.15.45:0/2030064871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7958101420 0x7f795810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 --2- 172.21.15.45:0/2030064871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f7958100ee0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 --2- 172.21.15.45:0/2030064871 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7958106a80 0x7f7958106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 -- 172.21.15.45:0/2030064871 >> 172.21.15.45:0/2030064871 conn(0x7f79580fc7a0 msgr2=0x7f79580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 -- 172.21.15.45:0/2030064871 shutdown_connections 2024-09-10T10:02:22.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.991+0000 7f795c813640 1 -- 172.21.15.45:0/2030064871 wait complete. 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 Processor -- start 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 -- start start 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f79581984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7958101420 0x7f7958198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7958106a80 0x7f795819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7958110ea0 con 0x7f7958100a80 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7958110d20 con 0x7f7958101420 2024-09-10T10:02:22.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.992+0000 7f795c813640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7958111020 con 0x7f7958106a80 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f79581984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f79581984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52478/0 (socket says 172.21.15.45:52478) 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 -- 172.21.15.45:0/4066385405 learned_addr learned my addr 172.21.15.45:0/4066385405 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f7955d9b640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7958101420 0x7f7958198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f7956d9d640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7958106a80 0x7f795819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 -- 172.21.15.45:0/4066385405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7958106a80 msgr2=0x7f795819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7958106a80 0x7f795819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 -- 172.21.15.45:0/4066385405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7958101420 msgr2=0x7f7958198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7958101420 0x7f7958198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 -- 172.21.15.45:0/4066385405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795819d480 con 0x7f7958100a80 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795659c640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f79581984d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f794800b4a0 tx=0x7f794800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f79477fe640 1 -- 172.21.15.45:0/4066385405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948014070 con 0x7f7958100a80 2024-09-10T10:02:22.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f79477fe640 1 -- 172.21.15.45:0/4066385405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7948010af0 con 0x7f7958100a80 2024-09-10T10:02:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f79477fe640 1 -- 172.21.15.45:0/4066385405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948017d10 con 0x7f7958100a80 2024-09-10T10:02:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795819d770 con 0x7f7958100a80 2024-09-10T10:02:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.993+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79581a5050 con 0x7f7958100a80 2024-09-10T10:02:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.995+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7958100ee0 con 0x7f7958100a80 2024-09-10T10:02:22.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.996+0000 7f79477fe640 1 -- 172.21.15.45:0/4066385405 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7948017070 con 0x7f7958100a80 2024-09-10T10:02:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.996+0000 7f79477fe640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f79400a2c10 0x7f79400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.997+0000 7f7955d9b640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f79400a2c10 0x7f79400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.997+0000 7f79477fe640 1 -- 172.21.15.45:0/4066385405 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f79480ce870 con 0x7f7958100a80 2024-09-10T10:02:22.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:22.997+0000 7f7955d9b640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f79400a2c10 0x7f79400a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f794c009900 tx=0x7f794c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:23.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.002+0000 7f79477fe640 1 -- 172.21.15.45:0/4066385405 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f794809b730 con 0x7f7958100a80 2024-09-10T10:02:23.064 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:23.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.185+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7958199830 con 0x7f7958100a80 2024-09-10T10:02:23.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.185+0000 7f79477fe640 1 -- 172.21.15.45:0/4066385405 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7958199830 con 0x7f7958100a80 2024-09-10T10:02:23.187 INFO:teuthology.orchestra.run.smithi045.stdout:137438953529 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f79400a2c10 msgr2=0x7f79400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f79400a2c10 0x7f79400a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f794c009900 tx=0x7f794c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 msgr2=0x7f79581984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f79581984d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f794800b4a0 tx=0x7f794800b960 comp rx=0 tx=0).stop 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 shutdown_connections 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f79400a2c10 0x7f79400a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7958106a80 0x7f795819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7958101420 0x7f7958198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 --2- 172.21.15.45:0/4066385405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7958100a80 0x7f79581984d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 >> 172.21.15.45:0/4066385405 conn(0x7f79580fc7a0 msgr2=0x7f795806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:23.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 shutdown_connections 2024-09-10T10:02:23.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:23.187+0000 7f795c813640 1 -- 172.21.15.45:0/4066385405 wait complete. 2024-09-10T10:02:23.293 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:23 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/4066385405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:02:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:23 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/4066385405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:02:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:23 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/4066385405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T10:02:23.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574920 for osd.1 2024-09-10T10:02:23.819 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:24.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:24 smithi162 ceph-mon[34963]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:24.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:24 smithi045 ceph-mon[31114]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:24.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:24 smithi047 ceph-mon[35244]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:24.957 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953527 got 137438953529 for osd.4 2024-09-10T10:02:24.957 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:25.097 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:25.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.096+0000 7fd95f3a2640 1 -- 172.21.15.45:0/3804856013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9581008d0 msgr2=0x7fd958100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.096+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/3804856013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9581008d0 0x7fd958100d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd948009930 tx=0x7fd94802a420 comp rx=0 tx=0).stop 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.096+0000 7fd95f3a2640 1 -- 172.21.15.45:0/3804856013 shutdown_connections 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.096+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/3804856013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd958101270 0x7fd95810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.096+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/3804856013 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9581008d0 0x7fd958100d30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.096+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/3804856013 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9581068d0 0x7fd958106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.097+0000 7fd95f3a2640 1 -- 172.21.15.45:0/3804856013 >> 172.21.15.45:0/3804856013 conn(0x7fd9580fc610 msgr2=0x7fd9580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.097+0000 7fd95f3a2640 1 -- 172.21.15.45:0/3804856013 shutdown_connections 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.097+0000 7fd95f3a2640 1 -- 172.21.15.45:0/3804856013 wait complete. 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.097+0000 7fd95f3a2640 1 Processor -- start 2024-09-10T10:02:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95f3a2640 1 -- start start 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95f3a2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9581008d0 0x7fd9581982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95f3a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd958101270 0x7fd958198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95f3a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9581068d0 0x7fd95819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95f3a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd958110cf0 con 0x7fd958101270 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95f3a2640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd958110b70 con 0x7fd9581008d0 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95f3a2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd958110e70 con 0x7fd9581068d0 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd958101270 0x7fd958198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd958101270 0x7fd958198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48834/0 (socket says 172.21.15.45:48834) 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 -- 172.21.15.45:0/2212122956 learned_addr learned my addr 172.21.15.45:0/2212122956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:25.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95e3a0640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9581008d0 0x7fd9581982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95eba1640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9581068d0 0x7fd95819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 -- 172.21.15.45:0/2212122956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9581068d0 msgr2=0x7fd95819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9581068d0 0x7fd95819cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 -- 172.21.15.45:0/2212122956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9581008d0 msgr2=0x7fd9581982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9581008d0 0x7fd9581982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.098+0000 7fd95db9f640 1 -- 172.21.15.45:0/2212122956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95819d270 con 0x7fd958101270 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.099+0000 7fd95db9f640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd958101270 0x7fd958198800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd948009a50 tx=0x7fd948004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.099+0000 7fd94f7fe640 1 -- 172.21.15.45:0/2212122956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94803f030 con 0x7fd958101270 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.099+0000 7fd94f7fe640 1 -- 172.21.15.45:0/2212122956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd948003ad0 con 0x7fd958101270 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.099+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95819d500 con 0x7fd958101270 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.099+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9581a4e40 con 0x7fd958101270 2024-09-10T10:02:25.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.099+0000 7fd94f7fe640 1 -- 172.21.15.45:0/2212122956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948035070 con 0x7fd958101270 2024-09-10T10:02:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.100+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd958100d30 con 0x7fd958101270 2024-09-10T10:02:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.101+0000 7fd94f7fe640 1 -- 172.21.15.45:0/2212122956 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd948042030 con 0x7fd958101270 2024-09-10T10:02:25.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.102+0000 7fd94f7fe640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd9440a2c90 0x7fd9440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.102+0000 7fd95e3a0640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd9440a2c90 0x7fd9440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.102+0000 7fd94f7fe640 1 -- 172.21.15.45:0/2212122956 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd9480efd80 con 0x7fd958101270 2024-09-10T10:02:25.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.103+0000 7fd95e3a0640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd9440a2c90 0x7fd9440a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd95400abe0 tx=0x7fd95400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.108+0000 7fd94f7fe640 1 -- 172.21.15.45:0/2212122956 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9480bcb30 con 0x7fd958101270 2024-09-10T10:02:25.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.295+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd958199620 con 0x7fd958101270 2024-09-10T10:02:25.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.295+0000 7fd94f7fe640 1 -- 172.21.15.45:0/2212122956 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd958199620 con 0x7fd958101270 2024-09-10T10:02:25.297 INFO:teuthology.orchestra.run.smithi045.stdout:322122547223 2024-09-10T10:02:25.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd9440a2c90 msgr2=0x7fd9440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd9440a2c90 0x7fd9440a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd95400abe0 tx=0x7fd95400a2d0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd958101270 msgr2=0x7fd958198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd958101270 0x7fd958198800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd948009a50 tx=0x7fd948004270 comp rx=0 tx=0).stop 2024-09-10T10:02:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 shutdown_connections 2024-09-10T10:02:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd9440a2c90 0x7fd9440a5150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd9581068d0 0x7fd95819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd958101270 0x7fd958198800 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 --2- 172.21.15.45:0/2212122956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9581008d0 0x7fd9581982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 >> 172.21.15.45:0/2212122956 conn(0x7fd9580fc610 msgr2=0x7fd95806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 shutdown_connections 2024-09-10T10:02:25.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.297+0000 7fd95f3a2640 1 -- 172.21.15.45:0/2212122956 wait complete. 2024-09-10T10:02:25.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:25 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2212122956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:02:25.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:25 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2212122956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:02:25.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:25 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2212122956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T10:02:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 -- 172.21.15.45:0/429885431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a010c160 msgr2=0x7f44a010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 --2- 172.21.15.45:0/429885431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a010c160 0x7f44a010e600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4498009ca0 tx=0x7f449802a950 comp rx=0 tx=0).stop 2024-09-10T10:02:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 -- 172.21.15.45:0/429885431 shutdown_connections 2024-09-10T10:02:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 --2- 172.21.15.45:0/429885431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a010c160 0x7f44a010e600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 --2- 172.21.15.45:0/429885431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a0103650 0x7f44a010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 --2- 172.21.15.45:0/429885431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a0102d30 0x7f44a0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 -- 172.21.15.45:0/429885431 >> 172.21.15.45:0/429885431 conn(0x7f44a00fc7d0 msgr2=0x7f44a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.711+0000 7f44a6128640 1 -- 172.21.15.45:0/429885431 shutdown_connections 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.712+0000 7f44a6128640 1 -- 172.21.15.45:0/429885431 wait complete. 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.712+0000 7f44a6128640 1 Processor -- start 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.712+0000 7f44a6128640 1 -- start start 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f44a6128640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a0102d30 0x7f44a0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f44a6128640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a0103650 0x7f44a0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f44a6128640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a010c160 0x7f44a019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f44a6128640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0111f60 con 0x7f44a0102d30 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f44a6128640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f44a0111de0 con 0x7f44a010c160 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f44a6128640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01120e0 con 0x7f44a0103650 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a0102d30 0x7f44a0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a0102d30 0x7f44a0198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48852/0 (socket says 172.21.15.45:48852) 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 -- 172.21.15.45:0/2236931718 learned_addr learned my addr 172.21.15.45:0/2236931718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449ffff640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a010c160 0x7f44a019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449effd640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a0103650 0x7f44a0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 -- 172.21.15.45:0/2236931718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a0103650 msgr2=0x7f44a0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a0103650 0x7f44a0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 -- 172.21.15.45:0/2236931718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a010c160 msgr2=0x7f44a019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a010c160 0x7f44a019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 -- 172.21.15.45:0/2236931718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a019d3b0 con 0x7f44a0102d30 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449f7fe640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a0102d30 0x7f44a0198400 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f449000d770 tx=0x7f449000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449cff9640 1 -- 172.21.15.45:0/2236931718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490017070 con 0x7f44a0102d30 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.713+0000 7f449cff9640 1 -- 172.21.15.45:0/2236931718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4490005500 con 0x7f44a0102d30 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.714+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a019d6a0 con 0x7f44a0102d30 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.714+0000 7f449cff9640 1 -- 172.21.15.45:0/2236931718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4490004e00 con 0x7f44a0102d30 2024-09-10T10:02:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.714+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01a4f80 con 0x7f44a0102d30 2024-09-10T10:02:25.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.715+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a0103110 con 0x7f44a0102d30 2024-09-10T10:02:25.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.716+0000 7f449cff9640 1 -- 172.21.15.45:0/2236931718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4490004fa0 con 0x7f44a0102d30 2024-09-10T10:02:25.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.716+0000 7f449cff9640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f44880a2c90 0x7f44880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.717+0000 7f449effd640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f44880a2c90 0x7f44880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.717+0000 7f449cff9640 1 -- 172.21.15.45:0/2236931718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f44900ceb20 con 0x7f44a0102d30 2024-09-10T10:02:25.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.717+0000 7f449effd640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f44880a2c90 0x7f44880a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44a010d4f0 tx=0x7f449400b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.721+0000 7f449cff9640 1 -- 172.21.15.45:0/2236931718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f449009b2e0 con 0x7f44a0102d30 2024-09-10T10:02:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.901+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f44a0107200 con 0x7f44a0102d30 2024-09-10T10:02:25.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.902+0000 7f449cff9640 1 -- 172.21.15.45:0/2236931718 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f449009f5e0 con 0x7f44a0102d30 2024-09-10T10:02:25.903 INFO:teuthology.orchestra.run.smithi045.stdout:270582939680 2024-09-10T10:02:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f44880a2c90 msgr2=0x7f44880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f44880a2c90 0x7f44880a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44a010d4f0 tx=0x7f449400b2e0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a0102d30 msgr2=0x7f44a0198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a0102d30 0x7f44a0198400 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f449000d770 tx=0x7f449000dc30 comp rx=0 tx=0).stop 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 shutdown_connections 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f44880a2c90 0x7f44880a5150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f44a010c160 0x7f44a019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f44a0103650 0x7f44a0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 --2- 172.21.15.45:0/2236931718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44a0102d30 0x7f44a0198400 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 >> 172.21.15.45:0/2236931718 conn(0x7f44a00fc7d0 msgr2=0x7f44a00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 shutdown_connections 2024-09-10T10:02:25.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.904+0000 7f44a6128640 1 -- 172.21.15.45:0/2236931718 wait complete. 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 -- 172.21.15.45:0/2522629400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694102570 msgr2=0x7fe6941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 --2- 172.21.15.45:0/2522629400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694102570 0x7fe6941029b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe688009900 tx=0x7fe68802a480 comp rx=0 tx=0).stop 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 -- 172.21.15.45:0/2522629400 shutdown_connections 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 --2- 172.21.15.45:0/2522629400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe694102ef0 0x7fe69410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 --2- 172.21.15.45:0/2522629400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694102570 0x7fe6941029b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 --2- 172.21.15.45:0/2522629400 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe694100e20 0x7fe694101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 -- 172.21.15.45:0/2522629400 >> 172.21.15.45:0/2522629400 conn(0x7fe6940fc7d0 msgr2=0x7fe6940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 -- 172.21.15.45:0/2522629400 shutdown_connections 2024-09-10T10:02:25.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 -- 172.21.15.45:0/2522629400 wait complete. 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 Processor -- start 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.955+0000 7fe69bac5640 1 -- start start 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69bac5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694100e20 0x7fe694199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69bac5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe694102570 0x7fe6941998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69bac5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe694102ef0 0x7fe6941a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69bac5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe694110f80 con 0x7fe694100e20 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69bac5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe694110e00 con 0x7fe694102570 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69bac5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe694111100 con 0x7fe694102ef0 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694100e20 0x7fe694199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694100e20 0x7fe694199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48866/0 (socket says 172.21.15.45:48866) 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 -- 172.21.15.45:0/268533425 learned_addr learned my addr 172.21.15.45:0/268533425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:25.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699060640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe694102570 0x7fe6941998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 -- 172.21.15.45:0/268533425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe694102ef0 msgr2=0x7fe6941a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69a062640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe694102ef0 0x7fe6941a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe694102ef0 0x7fe6941a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 -- 172.21.15.45:0/268533425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe694102570 msgr2=0x7fe6941998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe694102570 0x7fe6941998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 -- 172.21.15.45:0/268533425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6941a4950 con 0x7fe694100e20 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69a062640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe694102ef0 0x7fe6941a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699060640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe694102570 0x7fe6941998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe699861640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694100e20 0x7fe694199390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c970 tx=0x7fe69000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe686ffd640 1 -- 172.21.15.45:0/268533425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690007bb0 con 0x7fe694100e20 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941a4b20 con 0x7fe694100e20 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe686ffd640 1 -- 172.21.15.45:0/268533425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe690007d50 con 0x7fe694100e20 2024-09-10T10:02:25.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.956+0000 7fe686ffd640 1 -- 172.21.15.45:0/268533425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69000f450 con 0x7fe694100e20 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.957+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a4fe0 con 0x7fe694100e20 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.957+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe694101200 con 0x7fe694100e20 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.959+0000 7fe686ffd640 1 -- 172.21.15.45:0/268533425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe690004fe0 con 0x7fe694100e20 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.959+0000 7fe686ffd640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe6800a2c10 0x7fe6800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.960+0000 7fe699060640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe6800a2c10 0x7fe6800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.960+0000 7fe686ffd640 1 -- 172.21.15.45:0/268533425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6900cf440 con 0x7fe694100e20 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.960+0000 7fe699060640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe6800a2c10 0x7fe6800a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe688000c00 tx=0x7fe6880041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:25.962+0000 7fe686ffd640 1 -- 172.21.15.45:0/268533425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe690014030 con 0x7fe694100e20 2024-09-10T10:02:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.139+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe69406a560 con 0x7fe694100e20 2024-09-10T10:02:26.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.139+0000 7fe686ffd640 1 -- 172.21.15.45:0/268533425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe69009c250 con 0x7fe694100e20 2024-09-10T10:02:26.141 INFO:teuthology.orchestra.run.smithi045.stdout:103079215167 2024-09-10T10:02:26.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe6800a2c10 msgr2=0x7fe6800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe6800a2c10 0x7fe6800a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe688000c00 tx=0x7fe6880041e0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694100e20 msgr2=0x7fe694199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694100e20 0x7fe694199390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c970 tx=0x7fe69000ce30 comp rx=0 tx=0).stop 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 shutdown_connections 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fe6800a2c10 0x7fe6800a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe694102ef0 0x7fe6941a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe694102570 0x7fe6941998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 --2- 172.21.15.45:0/268533425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe694100e20 0x7fe694199390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 >> 172.21.15.45:0/268533425 conn(0x7fe6940fc7d0 msgr2=0x7fe694105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.141+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 shutdown_connections 2024-09-10T10:02:26.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:26.142+0000 7fe69bac5640 1 -- 172.21.15.45:0/268533425 wait complete. 2024-09-10T10:02:26.388 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:26.389 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:26.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:26 smithi045 ceph-mon[31114]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:26.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:26 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2236931718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:02:26.506 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:26 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/268533425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:02:26.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:26 smithi162 ceph-mon[34963]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:26.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:26 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2236931718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:02:26.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:26 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/268533425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:02:26.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:26 smithi047 ceph-mon[35244]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:26.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:26 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2236931718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T10:02:26.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:26 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/268533425' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T10:02:26.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215164 got 103079215167 for osd.3 2024-09-10T10:02:26.904 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:26.955 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:26.955 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 -- 172.21.15.45:0/258983479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c101420 msgr2=0x7f0f3c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 --2- 172.21.15.45:0/258983479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c101420 0x7f0f3c10d950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009cf0 tx=0x7f0f3402cb00 comp rx=0 tx=0).stop 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 -- 172.21.15.45:0/258983479 shutdown_connections 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 --2- 172.21.15.45:0/258983479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c101420 0x7f0f3c10d950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 --2- 172.21.15.45:0/258983479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f3c100a80 0x7f0f3c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 --2- 172.21.15.45:0/258983479 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c106a80 0x7f0f3c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 -- 172.21.15.45:0/258983479 >> 172.21.15.45:0/258983479 conn(0x7f0f3c0fc7a0 msgr2=0x7f0f3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 -- 172.21.15.45:0/258983479 shutdown_connections 2024-09-10T10:02:28.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.193+0000 7f0f40f46640 1 -- 172.21.15.45:0/258983479 wait complete. 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.194+0000 7f0f40f46640 1 Processor -- start 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.194+0000 7f0f40f46640 1 -- start start 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.194+0000 7f0f40f46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c100a80 0x7f0f3c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.194+0000 7f0f40f46640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c101420 0x7f0f3c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f40f46640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f3c106a80 0x7f0f3c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f40f46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c10fd20 con 0x7f0f3c100a80 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f40f46640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c10fba0 con 0x7f0f3c106a80 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f40f46640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c10fea0 con 0x7f0f3c101420 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c100a80 0x7f0f3c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c100a80 0x7f0f3c1961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48886/0 (socket says 172.21.15.45:48886) 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 -- 172.21.15.45:0/2101068711 learned_addr learned my addr 172.21.15.45:0/2101068711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f39d9b640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c101420 0x7f0f3c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3ad9d640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f3c106a80 0x7f0f3c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 -- 172.21.15.45:0/2101068711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c101420 msgr2=0x7f0f3c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c101420 0x7f0f3c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 -- 172.21.15.45:0/2101068711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f3c106a80 msgr2=0x7f0f3c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f3c106a80 0x7f0f3c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 -- 172.21.15.45:0/2101068711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c19b190 con 0x7f0f3c100a80 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f3a59c640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c100a80 0x7f0f3c1961e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00c890 tx=0x7f0f2c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f2b7fe640 1 -- 172.21.15.45:0/2101068711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c004220 con 0x7f0f3c100a80 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f2b7fe640 1 -- 172.21.15.45:0/2101068711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c0043c0 con 0x7f0f3c100a80 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c19b480 con 0x7f0f3c100a80 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f2b7fe640 1 -- 172.21.15.45:0/2101068711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0049e0 con 0x7f0f3c100a80 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.195+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1a2d60 con 0x7f0f3c100a80 2024-09-10T10:02:28.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.196+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c100ee0 con 0x7f0f3c100a80 2024-09-10T10:02:28.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.198+0000 7f0f2b7fe640 1 -- 172.21.15.45:0/2101068711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0f2c0079b0 con 0x7f0f3c100a80 2024-09-10T10:02:28.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.198+0000 7f0f2b7fe640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0f240a2c10 0x7f0f240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:28.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.198+0000 7f0f2b7fe640 1 -- 172.21.15.45:0/2101068711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f0f2c0cf130 con 0x7f0f3c100a80 2024-09-10T10:02:28.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.198+0000 7f0f39d9b640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0f240a2c10 0x7f0f240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:28.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.199+0000 7f0f39d9b640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0f240a2c10 0x7f0f240a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c106110 tx=0x7f0f3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:28.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.202+0000 7f0f2b7fe640 1 -- 172.21.15.45:0/2101068711 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f2c014030 con 0x7f0f3c100a80 2024-09-10T10:02:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.379+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0f3c197540 con 0x7f0f3c100a80 2024-09-10T10:02:28.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.380+0000 7f0f2b7fe640 1 -- 172.21.15.45:0/2101068711 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0f2c09b8f0 con 0x7f0f3c100a80 2024-09-10T10:02:28.381 INFO:teuthology.orchestra.run.smithi045.stdout:188978561072 2024-09-10T10:02:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0f240a2c10 msgr2=0x7f0f240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0f240a2c10 0x7f0f240a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c106110 tx=0x7f0f3000b390 comp rx=0 tx=0).stop 2024-09-10T10:02:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c100a80 msgr2=0x7f0f3c1961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:28.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c100a80 0x7f0f3c1961e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00c890 tx=0x7f0f2c00cd50 comp rx=0 tx=0).stop 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 shutdown_connections 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f0f240a2c10 0x7f0f240a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f3c106a80 0x7f0f3c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0f3c101420 0x7f0f3c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 --2- 172.21.15.45:0/2101068711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f3c100a80 0x7f0f3c1961e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 >> 172.21.15.45:0/2101068711 conn(0x7f0f3c0fc7a0 msgr2=0x7f0f3c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 shutdown_connections 2024-09-10T10:02:28.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:28.382+0000 7f0f40f46640 1 -- 172.21.15.45:0/2101068711 wait complete. 2024-09-10T10:02:28.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:28 smithi045 ceph-mon[31114]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:28.634 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:28 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2101068711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:02:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:28 smithi162 ceph-mon[34963]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:28 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2101068711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:02:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:28 smithi047 ceph-mon[35244]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:28.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:28 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2101068711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T10:02:28.660 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939678 got 270582939680 for osd.9 2024-09-10T10:02:28.661 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:29.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 -- 172.21.15.45:0/4292153341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 msgr2=0x7ff3d01006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/4292153341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d01006f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009ca0 tx=0x7ff3c002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 -- 172.21.15.45:0/4292153341 shutdown_connections 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/4292153341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3d0106f30 0x7ff3d010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/4292153341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3d0100cc0 0x7ff3d0106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/4292153341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d01006f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 -- 172.21.15.45:0/4292153341 >> 172.21.15.45:0/4292153341 conn(0x7ff3d0077210 msgr2=0x7ff3d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 -- 172.21.15.45:0/4292153341 shutdown_connections 2024-09-10T10:02:29.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.148+0000 7ff3d4b93640 1 -- 172.21.15.45:0/4292153341 wait complete. 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 Processor -- start 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 -- start start 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3d0100cc0 0x7ff3d0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3d0106f30 0x7ff3d019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0111fb0 con 0x7ff3d0100310 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0111e30 con 0x7ff3d0106f30 2024-09-10T10:02:29.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3d4b93640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0112130 con 0x7ff3d0100cc0 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3ce59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3ce59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d0198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48912/0 (socket says 172.21.15.45:48912) 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.149+0000 7ff3ce59c640 1 -- 172.21.15.45:0/2296797026 learned_addr learned my addr 172.21.15.45:0/2296797026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3cdd9b640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3d0100cc0 0x7ff3d0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3ced9d640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3d0106f30 0x7ff3d019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3ce59c640 1 -- 172.21.15.45:0/2296797026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3d0100cc0 msgr2=0x7ff3d0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3ce59c640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3d0100cc0 0x7ff3d0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3ce59c640 1 -- 172.21.15.45:0/2296797026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3d0106f30 msgr2=0x7ff3d019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3ce59c640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3d0106f30 0x7ff3d019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3ce59c640 1 -- 172.21.15.45:0/2296797026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d019d4c0 con 0x7ff3d0100310 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3cdd9b640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3d0100cc0 0x7ff3d0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:29.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3ce59c640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d0198510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff3c002ade0 tx=0x7ff3c00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3bf7fe640 1 -- 172.21.15.45:0/2296797026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c002fa20 con 0x7ff3d0100310 2024-09-10T10:02:29.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3bf7fe640 1 -- 172.21.15.45:0/2296797026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c002fbc0 con 0x7ff3d0100310 2024-09-10T10:02:29.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d019d750 con 0x7ff3d0100310 2024-09-10T10:02:29.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3bf7fe640 1 -- 172.21.15.45:0/2296797026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00467b0 con 0x7ff3d0100310 2024-09-10T10:02:29.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.150+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01a5090 con 0x7ff3d0100310 2024-09-10T10:02:29.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.151+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d01006f0 con 0x7ff3d0100310 2024-09-10T10:02:29.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.152+0000 7ff3bf7fe640 1 -- 172.21.15.45:0/2296797026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff3c0046950 con 0x7ff3d0100310 2024-09-10T10:02:29.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.153+0000 7ff3bf7fe640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff3b80a2c10 0x7ff3b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.153+0000 7ff3bf7fe640 1 -- 172.21.15.45:0/2296797026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff3c00f0660 con 0x7ff3d0100310 2024-09-10T10:02:29.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.153+0000 7ff3cdd9b640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff3b80a2c10 0x7ff3b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.154+0000 7ff3cdd9b640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff3b80a2c10 0x7ff3b80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3d01048d0 tx=0x7ff3c4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.157+0000 7ff3bf7fe640 1 -- 172.21.15.45:0/2296797026 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3c00453e0 con 0x7ff3d0100310 2024-09-10T10:02:29.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.335+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff3d010a110 con 0x7ff3d0100310 2024-09-10T10:02:29.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.335+0000 7ff3bf7fe640 1 -- 172.21.15.45:0/2296797026 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff3c00bd610 con 0x7ff3d0100310 2024-09-10T10:02:29.337 INFO:teuthology.orchestra.run.smithi045.stdout:292057776155 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff3b80a2c10 msgr2=0x7ff3b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff3b80a2c10 0x7ff3b80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3d01048d0 tx=0x7ff3c4008040 comp rx=0 tx=0).stop 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 msgr2=0x7ff3d0198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d0198510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff3c002ade0 tx=0x7ff3c00319f0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 shutdown_connections 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ff3b80a2c10 0x7ff3b80a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff3d0106f30 0x7ff3d019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3d0100cc0 0x7ff3d0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7ff3d4b93640 1 --2- 172.21.15.45:0/2296797026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3d0100310 0x7ff3d0198510 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 >> 172.21.15.45:0/2296797026 conn(0x7ff3d0077210 msgr2=0x7ff3d00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 shutdown_connections 2024-09-10T10:02:29.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7ff3d4b93640 1 -- 172.21.15.45:0/2296797026 wait complete. 2024-09-10T10:02:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7f898f6f3640 1 -- 172.21.15.45:0/608589329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 msgr2=0x7f898810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.337+0000 7f898f6f3640 1 --2- 172.21.15.45:0/608589329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898810ec40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8980009cd0 tx=0x7f898002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7f898f6f3640 1 -- 172.21.15.45:0/608589329 shutdown_connections 2024-09-10T10:02:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7f898f6f3640 1 --2- 172.21.15.45:0/608589329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898810ec40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7f898f6f3640 1 --2- 172.21.15.45:0/608589329 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f89881036e0 0x7f8988107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7f898f6f3640 1 --2- 172.21.15.45:0/608589329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8988102d30 0x7f8988103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7f898f6f3640 1 -- 172.21.15.45:0/608589329 >> 172.21.15.45:0/608589329 conn(0x7f89880fc7d0 msgr2=0x7f89880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7f898f6f3640 1 -- 172.21.15.45:0/608589329 shutdown_connections 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.338+0000 7f898f6f3640 1 -- 172.21.15.45:0/608589329 wait complete. 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 Processor -- start 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 -- start start 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988102d30 0x7f8988198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89881036e0 0x7f8988198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89881120a0 con 0x7f89881082a0 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8988111f20 con 0x7f89881036e0 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898f6f3640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8988112220 con 0x7f8988102d30 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898dc90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898dc90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48924/0 (socket says 172.21.15.45:48924) 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898dc90640 1 -- 172.21.15.45:0/2937740484 learned_addr learned my addr 172.21.15.45:0/2937740484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898cc8e640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89881036e0 0x7f8988198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898d48f640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988102d30 0x7f8988198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.339+0000 7f898dc90640 1 -- 172.21.15.45:0/2937740484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988102d30 msgr2=0x7f8988198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898dc90640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988102d30 0x7f8988198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898dc90640 1 -- 172.21.15.45:0/2937740484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89881036e0 msgr2=0x7f8988198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898dc90640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89881036e0 0x7f8988198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898dc90640 1 -- 172.21.15.45:0/2937740484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898819d4f0 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898d48f640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988102d30 0x7f8988198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898cc8e640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89881036e0 0x7f8988198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898dc90640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898819ce10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8980005cc0 tx=0x7f8980003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f897e7fc640 1 -- 172.21.15.45:0/2937740484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980004040 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898819d780 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f897e7fc640 1 -- 172.21.15.45:0/2937740484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89800046c0 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f897e7fc640 1 -- 172.21.15.45:0/2937740484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980046480 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.340+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89881a50c0 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.341+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8988103110 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.342+0000 7f897e7fc640 1 -- 172.21.15.45:0/2937740484 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f89800305a0 con 0x7f89881082a0 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.343+0000 7f897e7fc640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f89740a2c90 0x7f89740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.343+0000 7f898d48f640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f89740a2c90 0x7f89740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.343+0000 7f897e7fc640 1 -- 172.21.15.45:0/2937740484 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f898002e030 con 0x7f89881082a0 2024-09-10T10:02:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.344+0000 7f898d48f640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f89740a2c90 0x7f89740a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8984009b40 tx=0x7f8984005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.347+0000 7f897e7fc640 1 -- 172.21.15.45:0/2937740484 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f898004b060 con 0x7f89881082a0 2024-09-10T10:02:29.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.531+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f89881a5290 con 0x7f89881082a0 2024-09-10T10:02:29.533 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:29 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2296797026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:02:29.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.532+0000 7f897e7fc640 1 -- 172.21.15.45:0/2937740484 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f89800bd5f0 con 0x7f89881082a0 2024-09-10T10:02:29.533 INFO:teuthology.orchestra.run.smithi045.stdout:158913790005 2024-09-10T10:02:29.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f89740a2c90 msgr2=0x7f89740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f89740a2c90 0x7f89740a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8984009b40 tx=0x7f8984005e50 comp rx=0 tx=0).stop 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 msgr2=0x7f898819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898819ce10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8980005cc0 tx=0x7f8980003970 comp rx=0 tx=0).stop 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 shutdown_connections 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f89740a2c90 0x7f89740a5150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89881082a0 0x7f898819ce10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89881036e0 0x7f8988198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 --2- 172.21.15.45:0/2937740484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8988102d30 0x7f8988198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 >> 172.21.15.45:0/2937740484 conn(0x7f89880fc7d0 msgr2=0x7f8988105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 shutdown_connections 2024-09-10T10:02:29.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:29.534+0000 7f898f6f3640 1 -- 172.21.15.45:0/2937740484 wait complete. 2024-09-10T10:02:29.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:29 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2296797026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:02:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:29 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2296797026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T10:02:29.938 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547221 got 322122547223 for osd.11 2024-09-10T10:02:29.938 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:30.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 -- 172.21.15.45:0/1665956778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4106f10 msgr2=0x7f12e410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 --2- 172.21.15.45:0/1665956778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4106f10 0x7f12e410ea90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009ca0 tx=0x7f12dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 -- 172.21.15.45:0/1665956778 shutdown_connections 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 --2- 172.21.15.45:0/1665956778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4106f10 0x7f12e410ea90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 --2- 172.21.15.45:0/1665956778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12e4102790 0x7f12e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 --2- 172.21.15.45:0/1665956778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e4101de0 0x7f12e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 -- 172.21.15.45:0/1665956778 >> 172.21.15.45:0/1665956778 conn(0x7f12e40fd8c0 msgr2=0x7f12e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 -- 172.21.15.45:0/1665956778 shutdown_connections 2024-09-10T10:02:30.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.214+0000 7f12ead65640 1 -- 172.21.15.45:0/1665956778 wait complete. 2024-09-10T10:02:30.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 Processor -- start 2024-09-10T10:02:30.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 -- start start 2024-09-10T10:02:30.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e4101de0 0x7f12e4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4102790 0x7f12e41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12e4106f10 0x7f12e419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4111fe0 con 0x7f12e4102790 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4111e60 con 0x7f12e4101de0 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12ead65640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4112160 con 0x7f12e4106f10 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4102790 0x7f12e41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4102790 0x7f12e41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48948/0 (socket says 172.21.15.45:48948) 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12e3fff640 1 -- 172.21.15.45:0/3633608691 learned_addr learned my addr 172.21.15.45:0/3633608691 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.215+0000 7f12e8b01640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e4101de0 0x7f12e4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e9302640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12e4106f10 0x7f12e419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e3fff640 1 -- 172.21.15.45:0/3633608691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12e4106f10 msgr2=0x7f12e419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e3fff640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12e4106f10 0x7f12e419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e3fff640 1 -- 172.21.15.45:0/3633608691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e4101de0 msgr2=0x7f12e4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e3fff640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e4101de0 0x7f12e4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e3fff640 1 -- 172.21.15.45:0/3633608691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e419d410 con 0x7f12e4102790 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e9302640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12e4106f10 0x7f12e419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e8b01640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e4101de0 0x7f12e4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e3fff640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4102790 0x7f12e41989a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f12d000b4a0 tx=0x7f12d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e1ffb640 1 -- 172.21.15.45:0/3633608691 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0014070 con 0x7f12e4102790 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e419d700 con 0x7f12e4102790 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e1ffb640 1 -- 172.21.15.45:0/3633608691 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d0010af0 con 0x7f12e4102790 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12e1ffb640 1 -- 172.21.15.45:0/3633608691 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0017d10 con 0x7f12e4102790 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.216+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e41a4fe0 con 0x7f12e4102790 2024-09-10T10:02:30.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.217+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e41021c0 con 0x7f12e4102790 2024-09-10T10:02:30.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.218+0000 7f12e1ffb640 1 -- 172.21.15.45:0/3633608691 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f12d0017070 con 0x7f12e4102790 2024-09-10T10:02:30.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.219+0000 7f12e1ffb640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f12d40a2c10 0x7f12d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:30.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.219+0000 7f12e8b01640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f12d40a2c10 0x7f12d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:30.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.219+0000 7f12e1ffb640 1 -- 172.21.15.45:0/3633608691 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f12d00ce530 con 0x7f12e4102790 2024-09-10T10:02:30.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.220+0000 7f12e8b01640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f12d40a2c10 0x7f12d40a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f12d80058c0 tx=0x7f12d8005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:30.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.223+0000 7f12e1ffb640 1 -- 172.21.15.45:0/3633608691 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12d009b450 con 0x7f12e4102790 2024-09-10T10:02:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.405+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f12e410a150 con 0x7f12e4102790 2024-09-10T10:02:30.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.406+0000 7f12e1ffb640 1 -- 172.21.15.45:0/3633608691 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f12d009ad50 con 0x7f12e4102790 2024-09-10T10:02:30.407 INFO:teuthology.orchestra.run.smithi045.stdout:77309411395 2024-09-10T10:02:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f12d40a2c10 msgr2=0x7f12d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f12d40a2c10 0x7f12d40a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f12d80058c0 tx=0x7f12d8005ec0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4102790 msgr2=0x7f12e41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4102790 0x7f12e41989a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f12d000b4a0 tx=0x7f12d000b960 comp rx=0 tx=0).stop 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 shutdown_connections 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f12d40a2c10 0x7f12d40a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12e4106f10 0x7f12e419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12e4102790 0x7f12e41989a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 --2- 172.21.15.45:0/3633608691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12e4101de0 0x7f12e4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 >> 172.21.15.45:0/3633608691 conn(0x7f12e40fd8c0 msgr2=0x7f12e40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 shutdown_connections 2024-09-10T10:02:30.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:30.408+0000 7f12ead65640 1 -- 172.21.15.45:0/3633608691 wait complete. 2024-09-10T10:02:30.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:30 smithi045 ceph-mon[31114]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:30.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:30 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2937740484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:02:30.557 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:30 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:30.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:30 smithi047 ceph-mon[35244]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:30.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:30 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2937740484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:02:30.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:30 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:30.801 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:30 smithi162 ceph-mon[34963]: pgmap v303: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:30.801 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:30 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2937740484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T10:02:30.801 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:30 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:30.922 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561070 got 188978561072 for osd.6 2024-09-10T10:02:30.922 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:31.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776155 for osd.10 2024-09-10T10:02:31.358 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:31.578 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:31 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3633608691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:02:31.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:31 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3633608691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:02:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:31 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3633608691' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T10:02:32.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790002 got 158913790005 for osd.5 2024-09-10T10:02:32.197 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 -- 172.21.15.45:0/1279965562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641036e0 msgr2=0x7f1264107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 --2- 172.21.15.45:0/1279965562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641036e0 0x7f1264107b90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1258009930 tx=0x7f125802a420 comp rx=0 tx=0).stop 2024-09-10T10:02:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 -- 172.21.15.45:0/1279965562 shutdown_connections 2024-09-10T10:02:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 --2- 172.21.15.45:0/1279965562 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12641082a0 0x7f126410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 --2- 172.21.15.45:0/1279965562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641036e0 0x7f1264107b90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 --2- 172.21.15.45:0/1279965562 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1264102d30 0x7f1264103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 -- 172.21.15.45:0/1279965562 >> 172.21.15.45:0/1279965562 conn(0x7f12640fc7d0 msgr2=0x7f12640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 -- 172.21.15.45:0/1279965562 shutdown_connections 2024-09-10T10:02:32.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.383+0000 7f126c0bc640 1 -- 172.21.15.45:0/1279965562 wait complete. 2024-09-10T10:02:32.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 Processor -- start 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 -- start start 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1264102d30 0x7f1264100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12641036e0 0x7f1264100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641082a0 0x7f1264101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f12641120b0 con 0x7f12641082a0 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1264111f30 con 0x7f1264102d30 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126c0bc640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1264112230 con 0x7f12641036e0 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641082a0 0x7f1264101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f1269e58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1264102d30 0x7f1264100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641082a0 0x7f1264101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48968/0 (socket says 172.21.15.45:48968) 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f1269e58640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1264102d30 0x7f1264100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:41056/0 (socket says 172.21.15.45:41056) 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 -- 172.21.15.45:0/2352633129 learned_addr learned my addr 172.21.15.45:0/2352633129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:32.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f1269657640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12641036e0 0x7f1264100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 -- 172.21.15.45:0/2352633129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12641036e0 msgr2=0x7f1264100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12641036e0 0x7f1264100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 -- 172.21.15.45:0/2352633129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1264102d30 msgr2=0x7f1264100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1264102d30 0x7f1264100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.384+0000 7f126a659640 1 -- 172.21.15.45:0/2352633129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12641a6a70 con 0x7f12641082a0 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.385+0000 7f126a659640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641082a0 0x7f1264101410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f125400eb10 tx=0x7f125400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.385+0000 7f1262ffd640 1 -- 172.21.15.45:0/2352633129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254004b20 con 0x7f12641082a0 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.385+0000 7f1262ffd640 1 -- 172.21.15.45:0/2352633129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1254004cc0 con 0x7f12641082a0 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.385+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12641a6d60 con 0x7f12641082a0 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.385+0000 7f1262ffd640 1 -- 172.21.15.45:0/2352633129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254010760 con 0x7f12641082a0 2024-09-10T10:02:32.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.385+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12641a7220 con 0x7f12641082a0 2024-09-10T10:02:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.386+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1264103110 con 0x7f12641082a0 2024-09-10T10:02:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.388+0000 7f1262ffd640 1 -- 172.21.15.45:0/2352633129 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1254005020 con 0x7f12641082a0 2024-09-10T10:02:32.389 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.389+0000 7f1262ffd640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f124c0a2c10 0x7f124c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.389+0000 7f1262ffd640 1 -- 172.21.15.45:0/2352633129 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f12540cf710 con 0x7f12641082a0 2024-09-10T10:02:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.389+0000 7f1269e58640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f124c0a2c10 0x7f124c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:32.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.389+0000 7f1269e58640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f124c0a2c10 0x7f124c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f125c009bd0 tx=0x7f125c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.394+0000 7f1262ffd640 1 -- 172.21.15.45:0/2352633129 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f125409c4c0 con 0x7f12641082a0 2024-09-10T10:02:32.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:32 smithi045 ceph-mon[31114]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-10T10:02:32.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.511 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.512 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:32 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.574+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f126410a2b0 con 0x7f12641082a0 2024-09-10T10:02:32.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.574+0000 7f1262ffd640 1 -- 172.21.15.45:0/2352633129 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f125409bdc0 con 0x7f12641082a0 2024-09-10T10:02:32.575 INFO:teuthology.orchestra.run.smithi045.stdout:240518168615 2024-09-10T10:02:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f124c0a2c10 msgr2=0x7f124c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f124c0a2c10 0x7f124c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f125c009bd0 tx=0x7f125c005f90 comp rx=0 tx=0).stop 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641082a0 msgr2=0x7f1264101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641082a0 0x7f1264101410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f125400eb10 tx=0x7f125400c2d0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 shutdown_connections 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f124c0a2c10 0x7f124c0a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f12641082a0 0x7f1264101410 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f12641036e0 0x7f1264100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 --2- 172.21.15.45:0/2352633129 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1264102d30 0x7f1264100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 >> 172.21.15.45:0/2352633129 conn(0x7f12640fc7d0 msgr2=0x7f1264106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 shutdown_connections 2024-09-10T10:02:32.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:32.576+0000 7f126c0bc640 1 -- 172.21.15.45:0/2352633129 wait complete. 2024-09-10T10:02:32.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:32 smithi047 ceph-mon[35244]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-10T10:02:32.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:32 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.854 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411392 got 77309411395 for osd.2 2024-09-10T10:02:32.854 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:32.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:32 smithi162 ceph-mon[34963]: pgmap v304: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-10T10:02:32.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:32.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:32 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:33.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:33 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2352633129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:02:33.707 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168610 got 240518168615 for osd.8 2024-09-10T10:02:33.708 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:33 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2352633129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:02:33.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:33 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2352633129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 -- 172.21.15.45:0/3954973315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 msgr2=0x7f7a78100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 --2- 172.21.15.45:0/3954973315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a78100860 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009900 tx=0x7f7a7402a410 comp rx=0 tx=0).stop 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 -- 172.21.15.45:0/3954973315 shutdown_connections 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 --2- 172.21.15.45:0/3954973315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a781055b0 0x7f7a7810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 --2- 172.21.15.45:0/3954973315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a78100e30 0x7f7a78104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 --2- 172.21.15.45:0/3954973315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a78100860 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 -- 172.21.15.45:0/3954973315 >> 172.21.15.45:0/3954973315 conn(0x7f7a78077210 msgr2=0x7f7a780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 -- 172.21.15.45:0/3954973315 shutdown_connections 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.941+0000 7f7a803cf640 1 -- 172.21.15.45:0/3954973315 wait complete. 2024-09-10T10:02:33.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 Processor -- start 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 -- start start 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a780fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a78100e30 0x7f7a780fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a781055b0 0x7f7a780fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78110e50 con 0x7f7a78100480 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78110cd0 con 0x7f7a781055b0 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a803cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78110fd0 con 0x7f7a78100e30 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e16b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a780fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e16b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a780fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59094/0 (socket says 172.21.15.45:59094) 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e16b640 1 -- 172.21.15.45:0/171563260 learned_addr learned my addr 172.21.15.45:0/171563260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e96c640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a781055b0 0x7f7a780fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7d96a640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a78100e30 0x7f7a780fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e16b640 1 -- 172.21.15.45:0/171563260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a78100e30 msgr2=0x7f7a780fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e16b640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a78100e30 0x7f7a780fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e16b640 1 -- 172.21.15.45:0/171563260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a781055b0 msgr2=0x7f7a780fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.942+0000 7f7a7e16b640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a781055b0 0x7f7a780fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a7e16b640 1 -- 172.21.15.45:0/171563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a781a68b0 con 0x7f7a78100480 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a7e96c640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a781055b0 0x7f7a780fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a7e16b640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a780fc690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a74006010 tx=0x7f7a740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a6f7fe640 1 -- 172.21.15.45:0/171563260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7403f030 con 0x7f7a78100480 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a6f7fe640 1 -- 172.21.15.45:0/171563260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a74003ce0 con 0x7f7a78100480 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a781a6b40 con 0x7f7a78100480 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a6f7fe640 1 -- 172.21.15.45:0/171563260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a74035070 con 0x7f7a78100480 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.943+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a781a7070 con 0x7f7a78100480 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.944+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a78100860 con 0x7f7a78100480 2024-09-10T10:02:33.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.945+0000 7f7a6f7fe640 1 -- 172.21.15.45:0/171563260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a74042030 con 0x7f7a78100480 2024-09-10T10:02:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.946+0000 7f7a6f7fe640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.947+0000 7f7a6f7fe640 1 -- 172.21.15.45:0/171563260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7a740ef980 con 0x7f7a78100480 2024-09-10T10:02:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.947+0000 7f7a7d96a640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.948+0000 7f7a7d96a640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7a640a2c90 0x7f7a640a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005e10 tx=0x7f7a68005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:33.950+0000 7f7a6f7fe640 1 -- 172.21.15.45:0/171563260 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a740bc7f0 con 0x7f7a78100480 2024-09-10T10:02:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.144+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7a781097a0 con 0x7f7a78100480 2024-09-10T10:02:34.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.144+0000 7f7a6f7fe640 1 -- 172.21.15.45:0/171563260 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a740bc0f0 con 0x7f7a78100480 2024-09-10T10:02:34.146 INFO:teuthology.orchestra.run.smithi045.stdout:210453397548 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7a640a2c90 msgr2=0x7f7a640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7a640a2c90 0x7f7a640a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005e10 tx=0x7f7a68005da0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 msgr2=0x7f7a780fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a780fc690 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7a74006010 tx=0x7f7a740042f0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 shutdown_connections 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7a640a2c90 0x7f7a640a5150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7a781055b0 0x7f7a780fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7a78100e30 0x7f7a780fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 --2- 172.21.15.45:0/171563260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a78100480 0x7f7a780fc690 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 >> 172.21.15.45:0/171563260 conn(0x7f7a78077210 msgr2=0x7f7a7810dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 shutdown_connections 2024-09-10T10:02:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:34.147+0000 7f7a803cf640 1 -- 172.21.15.45:0/171563260 wait complete. 2024-09-10T10:02:34.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:34 smithi045 ceph-mon[31114]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:34.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:34 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/171563260' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:02:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:34 smithi162 ceph-mon[34963]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:34.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:34 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/171563260' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:02:34.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:34 smithi047 ceph-mon[35244]: pgmap v305: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:34.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:34 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/171563260' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T10:02:35.012 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397546 got 210453397548 for osd.7 2024-09-10T10:02:35.012 DEBUG:teuthology.parallel:result is None 2024-09-10T10:02:35.012 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T10:02:35.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:02:35.417 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 -- 172.21.15.45:0/1527324330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104106a80 msgr2=0x7fd104106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 --2- 172.21.15.45:0/1527324330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104106a80 0x7fd104106e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009930 tx=0x7fd0f402a420 comp rx=0 tx=0).stop 2024-09-10T10:02:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 -- 172.21.15.45:0/1527324330 shutdown_connections 2024-09-10T10:02:35.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 --2- 172.21.15.45:0/1527324330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd104101420 0x7fd10410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 --2- 172.21.15.45:0/1527324330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd104100a80 0x7fd104100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 --2- 172.21.15.45:0/1527324330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104106a80 0x7fd104106e60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 -- 172.21.15.45:0/1527324330 >> 172.21.15.45:0/1527324330 conn(0x7fd1040fc7a0 msgr2=0x7fd1040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 -- 172.21.15.45:0/1527324330 shutdown_connections 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.856+0000 7fd109a35640 1 -- 172.21.15.45:0/1527324330 wait complete. 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.857+0000 7fd109a35640 1 Processor -- start 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.857+0000 7fd109a35640 1 -- start start 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd109a35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104100a80 0x7fd104198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd109a35640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd104101420 0x7fd1041989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd109a35640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd104106a80 0x7fd10419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd109a35640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd104110e40 con 0x7fd104100a80 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd109a35640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd104110cc0 con 0x7fd104106a80 2024-09-10T10:02:35.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd109a35640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd104110fc0 con 0x7fd104101420 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104100a80 0x7fd104198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104100a80 0x7fd104198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59114/0 (socket says 172.21.15.45:59114) 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 -- 172.21.15.45:0/3672965690 learned_addr learned my addr 172.21.15.45:0/3672965690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd1037fe640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd104106a80 0x7fd10419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd1027fc640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd104101420 0x7fd1041989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 -- 172.21.15.45:0/3672965690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd104101420 msgr2=0x7fd1041989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd104101420 0x7fd1041989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 -- 172.21.15.45:0/3672965690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd104106a80 msgr2=0x7fd10419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd104106a80 0x7fd10419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:35.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 -- 172.21.15.45:0/3672965690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10419d440 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.858+0000 7fd102ffd640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104100a80 0x7fd104198490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009230 tx=0x7fd0f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.859+0000 7fd108a33640 1 -- 172.21.15.45:0/3672965690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f403f030 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.859+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10419d6d0 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.859+0000 7fd108a33640 1 -- 172.21.15.45:0/3672965690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4003ad0 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.859+0000 7fd108a33640 1 -- 172.21.15.45:0/3672965690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4035070 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.859+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a5010 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.860+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd104100ee0 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.861+0000 7fd108a33640 1 -- 172.21.15.45:0/3672965690 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd0f4042030 con 0x7fd104100a80 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.861+0000 7fd108a33640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.862+0000 7fd1027fc640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:35.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.862+0000 7fd108a33640 1 -- 172.21.15.45:0/3672965690 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd0f40ef620 con 0x7fd104100a80 2024-09-10T10:02:35.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.862+0000 7fd1027fc640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1041051b0 tx=0x7fd0f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:35.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:35.866+0000 7fd108a33640 1 -- 172.21.15.45:0/3672965690 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0f40bbde0 con 0x7fd104100a80 2024-09-10T10:02:36.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.050+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd104062630 con 0x7fd0ec0a2c10 2024-09-10T10:02:36.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.057+0000 7fd108a33640 1 -- 172.21.15.45:0/3672965690 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258638 (secure 0 0 0) 0x7fd104062630 con 0x7fd0ec0a2c10 2024-09-10T10:02:36.060 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:36.061 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-10T10:02:36.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.059+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd0ec0a2c10 msgr2=0x7fd0ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.059+0000 7fd109a35640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd1041051b0 tx=0x7fd0f800b410 comp rx=0 tx=0).stop 2024-09-10T10:02:36.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.059+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104100a80 msgr2=0x7fd104198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.059+0000 7fd109a35640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104100a80 0x7fd104198490 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009230 tx=0x7fd0f4004270 comp rx=0 tx=0).stop 2024-09-10T10:02:36.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 shutdown_connections 2024-09-10T10:02:36.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd0ec0a2c10 0x7fd0ec0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd104106a80 0x7fd10419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd104101420 0x7fd1041989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 --2- 172.21.15.45:0/3672965690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd104100a80 0x7fd104198490 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 >> 172.21.15.45:0/3672965690 conn(0x7fd1040fc7a0 msgr2=0x7fd10406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 shutdown_connections 2024-09-10T10:02:36.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:36.060+0000 7fd109a35640 1 -- 172.21.15.45:0/3672965690 wait complete. 2024-09-10T10:02:36.543 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:36 smithi045 ceph-mon[31114]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:36.543 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:36.543 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:36 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-10T10:02:34.628499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654219,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":251,"num_read_kb":240,"num_write":266,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357620,"kb_used_data":12504,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584204,"statfs":{"total":1151940427776,"available":1151574224896,"internally_reserved":0,"allocated":12804096,"data_stored":7215969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335498+0000","last_change":"2024-09-10T10:01:34.253963+0000","last_active":"2024-09-10T10:01:50.335498+0000","last_peered":"2024-09-10T10:01:50.335498+0000","last_clean":"2024-09-10T10:01:50.335498+0000","last_became_active":"2024-09-10T10:01:34.252547+0000","last_became_peered":"2024-09-10T10:01:34.252547+0000","last_unstale":"2024-09-10T10:01:50.335498+0000","last_undegraded":"2024-09-10T10:01:50.335498+0000","last_fullsized":"2024-09-10T10:01:50.335498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:55.761035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321219+0000","last_change":"2024-09-10T10:01:34.249329+0000","last_active":"2024-09-10T10:01:50.321219+0000","last_peered":"2024-09-10T10:01:50.321219+0000","last_clean":"2024-09-10T10:01:50.321219+0000","last_became_active":"2024-09-10T10:01:34.249036+0000","last_became_peered":"2024-09-10T10:01:34.249036+0000","last_unstale":"2024-09-10T10:01:50.321219+0000","last_undegraded":"2024-09-10T10:01:50.321219+0000","last_fullsized":"2024-09-10T10:01:50.321219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:26.244307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320430+0000","last_change":"2024-09-10T10:01:34.242770+0000","last_active":"2024-09-10T10:01:50.320430+0000","last_peered":"2024-09-10T10:01:50.320430+0000","last_clean":"2024-09-10T10:01:50.320430+0000","last_became_active":"2024-09-10T10:01:34.242465+0000","last_became_peered":"2024-09-10T10:01:34.242465+0000","last_unstale":"2024-09-10T10:01:50.320430+0000","last_undegraded":"2024-09-10T10:01:50.320430+0000","last_fullsized":"2024-09-10T10:01:50.320430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:32.776914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838909+0000","last_change":"2024-09-10T10:01:34.249110+0000","last_active":"2024-09-10T10:01:50.838909+0000","last_peered":"2024-09-10T10:01:50.838909+0000","last_clean":"2024-09-10T10:01:50.838909+0000","last_became_active":"2024-09-10T10:01:34.248758+0000","last_became_peered":"2024-09-10T10:01:34.248758+0000","last_unstale":"2024-09-10T10:01:50.838909+0000","last_undegraded":"2024-09-10T10:01:50.838909+0000","last_fullsized":"2024-09-10T10:01:50.838909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:51.622686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338095+0000","last_change":"2024-09-10T10:01:34.248799+0000","last_active":"2024-09-10T10:01:50.338095+0000","last_peered":"2024-09-10T10:01:50.338095+0000","last_clean":"2024-09-10T10:01:50.338095+0000","last_became_active":"2024-09-10T10:01:34.246797+0000","last_became_peered":"2024-09-10T10:01:34.246797+0000","last_unstale":"2024-09-10T10:01:50.338095+0000","last_undegraded":"2024-09-10T10:01:50.338095+0000","last_fullsized":"2024-09-10T10:01:50.338095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:33.887465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320541+0000","last_change":"2024-09-10T10:01:34.252139+0000","last_active":"2024-09-10T10:01:50.320541+0000","last_peered":"2024-09-10T10:01:50.320541+0000","last_clean":"2024-09-10T10:01:50.320541+0000","last_became_active":"2024-09-10T10:01:34.251823+0000","last_became_peered":"2024-09-10T10:01:34.251823+0000","last_unstale":"2024-09-10T10:01:50.320541+0000","last_undegraded":"2024-09-10T10:01:50.320541+0000","last_fullsized":"2024-09-10T10:01:50.320541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:31.642428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337590+0000","last_change":"2024-09-10T10:01:34.258097+0000","last_active":"2024-09-10T10:01:50.337590+0000","last_peered":"2024-09-10T10:01:50.337590+0000","last_clean":"2024-09-10T10:01:50.337590+0000","last_became_active":"2024-09-10T10:01:34.257717+0000","last_became_peered":"2024-09-10T10:01:34.257717+0000","last_unstale":"2024-09-10T10:01:50.337590+0000","last_undegraded":"2024-09-10T10:01:50.337590+0000","last_fullsized":"2024-09-10T10:01:50.337590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:16.280503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319894+0000","last_change":"2024-09-10T10:01:34.249901+0000","last_active":"2024-09-10T10:01:50.319894+0000","last_peered":"2024-09-10T10:01:50.319894+0000","last_clean":"2024-09-10T10:01:50.319894+0000","last_became_active":"2024-09-10T10:01:34.249361+0000","last_became_peered":"2024-09-10T10:01:34.249361+0000","last_unstale":"2024-09-10T10:01:50.319894+0000","last_undegraded":"2024-09-10T10:01:50.319894+0000","last_fullsized":"2024-09-10T10:01:50.319894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:42.282621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.711941+0000","last_change":"2024-09-10T10:01:50.711491+0000","last_active":"2024-09-10T10:01:50.711941+0000","last_peered":"2024-09-10T10:01:50.711941+0000","last_clean":"2024-09-10T10:01:50.711941+0000","last_became_active":"2024-09-10T10:01:50.711162+0000","last_became_peered":"2024-09-10T10:01:50.711162+0000","last_unstale":"2024-09-10T10:01:50.711941+0000","last_undegraded":"2024-09-10T10:01:50.711941+0000","last_fullsized":"2024-09-10T10:01:50.711941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:08.502816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320934+0000","last_change":"2024-09-10T10:01:34.252538+0000","last_active":"2024-09-10T10:01:50.320934+0000","last_peered":"2024-09-10T10:01:50.320934+0000","last_clean":"2024-09-10T10:01:50.320934+0000","last_became_active":"2024-09-10T10:01:34.251384+0000","last_became_peered":"2024-09-10T10:01:34.251384+0000","last_unstale":"2024-09-10T10:01:50.320934+0000","last_undegraded":"2024-09-10T10:01:50.320934+0000","last_fullsized":"2024-09-10T10:01:50.320934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:39.990996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338061+0000","last_change":"2024-09-10T10:01:34.254262+0000","last_active":"2024-09-10T10:01:50.338061+0000","last_peered":"2024-09-10T10:01:50.338061+0000","last_clean":"2024-09-10T10:01:50.338061+0000","last_became_active":"2024-09-10T10:01:34.253907+0000","last_became_peered":"2024-09-10T10:01:34.253907+0000","last_unstale":"2024-09-10T10:01:50.338061+0000","last_undegraded":"2024-09-10T10:01:50.338061+0000","last_fullsized":"2024-09-10T10:01:50.338061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:38.681367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838921+0000","last_change":"2024-09-10T10:01:34.252194+0000","last_active":"2024-09-10T10:01:50.838921+0000","last_peered":"2024-09-10T10:01:50.838921+0000","last_clean":"2024-09-10T10:01:50.838921+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.838921+0000","last_undegraded":"2024-09-10T10:01:50.838921+0000","last_fullsized":"2024-09-10T10:01:50.838921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:47.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332280+0000","last_change":"2024-09-10T10:01:34.251795+0000","last_active":"2024-09-10T10:01:50.332280+0000","last_peered":"2024-09-10T10:01:50.332280+0000","last_clean":"2024-09-10T10:01:50.332280+0000","last_became_active":"2024-09-10T10:01:34.251403+0000","last_became_peered":"2024-09-10T10:01:34.251403+0000","last_unstale":"2024-09-10T10:01:50.332280+0000","last_undegraded":"2024-09-10T10:01:50.332280+0000","last_fullsized":"2024-09-10T10:01:50.332280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:11.650068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321079+0000","last_change":"2024-09-10T10:01:34.252718+0000","last_active":"2024-09-10T10:01:50.321079+0000","last_peered":"2024-09-10T10:01:50.321079+0000","last_clean":"2024-09-10T10:01:50.321079+0000","last_became_active":"2024-09-10T10:01:34.252415+0000","last_became_peered":"2024-09-10T10:01:34.252415+0000","last_unstale":"2024-09-10T10:01:50.321079+0000","last_undegraded":"2024-09-10T10:01:50.321079+0000","last_fullsized":"2024-09-10T10:01:50.321079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:51.057191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320828+0000","last_change":"2024-09-10T10:01:34.252237+0000","last_active":"2024-09-10T10:01:50.320828+0000","last_peered":"2024-09-10T10:01:50.320828+0000","last_clean":"2024-09-10T10:01:50.320828+0000","last_became_active":"2024-09-10T10:01:34.251909+0000","last_became_peered":"2024-09-10T10:01:34.251909+0000","last_unstale":"2024-09-10T10:01:50.320828+0000","last_undegraded":"2024-09-10T10:01:50.320828+0000","last_fullsized":"2024-09-10T10:01:50.320828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:15.150503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318205+0000","last_change":"2024-09-10T10:01:34.259107+0000","last_active":"2024-09-10T10:01:50.318205+0000","last_peered":"2024-09-10T10:01:50.318205+0000","last_clean":"2024-09-10T10:01:50.318205+0000","last_became_active":"2024-09-10T10:01:34.258230+0000","last_became_peered":"2024-09-10T10:01:34.258230+0000","last_unstale":"2024-09-10T10:01:50.318205+0000","last_undegraded":"2024-09-10T10:01:50.318205+0000","last_fullsized":"2024-09-10T10:01:50.318205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:36.091358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335573+0000","last_change":"2024-09-10T10:01:34.254341+0000","last_active":"2024-09-10T10:01:50.335573+0000","last_peered":"2024-09-10T10:01:50.335573+0000","last_clean":"2024-09-10T10:01:50.335573+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.335573+0000","last_undegraded":"2024-09-10T10:01:50.335573+0000","last_fullsized":"2024-09-10T10:01:50.335573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:46.668715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317729+0000","last_change":"2024-09-10T10:01:34.247950+0000","last_active":"2024-09-10T10:01:50.317729+0000","last_peered":"2024-09-10T10:01:50.317729+0000","last_clean":"2024-09-10T10:01:50.317729+0000","last_became_active":"2024-09-10T10:01:34.247659+0000","last_became_peered":"2024-09-10T10:01:34.247659+0000","last_unstale":"2024-09-10T10:01:50.317729+0000","last_undegraded":"2024-09-10T10:01:50.317729+0000","last_fullsized":"2024-09-10T10:01:50.317729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:34.113605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333284+0000","last_change":"2024-09-10T10:01:34.248153+0000","last_active":"2024-09-10T10:01:50.333284+0000","last_peered":"2024-09-10T10:01:50.333284+0000","last_clean":"2024-09-10T10:01:50.333284+0000","last_became_active":"2024-09-10T10:01:34.247773+0000","last_became_peered":"2024-09-10T10:01:34.247773+0000","last_unstale":"2024-09-10T10:01:50.333284+0000","last_undegraded":"2024-09-10T10:01:50.333284+0000","last_fullsized":"2024-09-10T10:01:50.333284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:54.223519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335273+0000","last_change":"2024-09-10T10:01:34.252781+0000","last_active":"2024-09-10T10:01:50.335273+0000","last_peered":"2024-09-10T10:01:50.335273+0000","last_clean":"2024-09-10T10:01:50.335273+0000","last_became_active":"2024-09-10T10:01:34.252459+0000","last_became_peered":"2024-09-10T10:01:34.252459+0000","last_unstale":"2024-09-10T10:01:50.335273+0000","last_undegraded":"2024-09-10T10:01:50.335273+0000","last_fullsized":"2024-09-10T10:01:50.335273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:32.039769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332286+0000","last_change":"2024-09-10T10:01:34.251147+0000","last_active":"2024-09-10T10:01:50.332286+0000","last_peered":"2024-09-10T10:01:50.332286+0000","last_clean":"2024-09-10T10:01:50.332286+0000","last_became_active":"2024-09-10T10:01:34.250835+0000","last_became_peered":"2024-09-10T10:01:34.250835+0000","last_unstale":"2024-09-10T10:01:50.332286+0000","last_undegraded":"2024-09-10T10:01:50.332286+0000","last_fullsized":"2024-09-10T10:01:50.332286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:56.914798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318417+0000","last_change":"2024-09-10T10:01:34.259300+0000","last_active":"2024-09-10T10:01:50.318417+0000","last_peered":"2024-09-10T10:01:50.318417+0000","last_clean":"2024-09-10T10:01:50.318417+0000","last_became_active":"2024-09-10T10:01:34.258192+0000","last_became_peered":"2024-09-10T10:01:34.258192+0000","last_unstale":"2024-09-10T10:01:50.318417+0000","last_undegraded":"2024-09-10T10:01:50.318417+0000","last_fullsized":"2024-09-10T10:01:50.318417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:19.755696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319962+0000","last_change":"2024-09-10T10:01:34.244797+0000","last_active":"2024-09-10T10:01:50.319962+0000","last_peered":"2024-09-10T10:01:50.319962+0000","last_clean":"2024-09-10T10:01:50.319962+0000","last_became_active":"2024-09-10T10:01:34.244464+0000","last_became_peered":"2024-09-10T10:01:34.244464+0000","last_unstale":"2024-09-10T10:01:50.319962+0000","last_undegraded":"2024-09-10T10:01:50.319962+0000","last_fullsized":"2024-09-10T10:01:50.319962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:23.750063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710213+0000","last_change":"2024-09-10T10:01:34.248445+0000","last_active":"2024-09-10T10:01:50.710213+0000","last_peered":"2024-09-10T10:01:50.710213+0000","last_clean":"2024-09-10T10:01:50.710213+0000","last_became_active":"2024-09-10T10:01:34.248020+0000","last_became_peered":"2024-09-10T10:01:34.248020+0000","last_unstale":"2024-09-10T10:01:50.710213+0000","last_undegraded":"2024-09-10T10:01:50.710213+0000","last_fullsized":"2024-09-10T10:01:50.710213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:32.096061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339780+0000","last_change":"2024-09-10T10:01:50.339780+0000","last_active":"2024-09-10T10:01:50.339780+0000","last_peered":"2024-09-10T10:01:50.339780+0000","last_clean":"2024-09-10T10:01:50.339780+0000","last_became_active":"2024-09-10T10:01:50.339519+0000","last_became_peered":"2024-09-10T10:01:50.339519+0000","last_unstale":"2024-09-10T10:01:50.339780+0000","last_undegraded":"2024-09-10T10:01:50.339780+0000","last_fullsized":"2024-09-10T10:01:50.339780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:27.368296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333351+0000","last_change":"2024-09-10T10:01:34.244512+0000","last_active":"2024-09-10T10:01:50.333351+0000","last_peered":"2024-09-10T10:01:50.333351+0000","last_clean":"2024-09-10T10:01:50.333351+0000","last_became_active":"2024-09-10T10:01:34.244191+0000","last_became_peered":"2024-09-10T10:01:34.244191+0000","last_unstale":"2024-09-10T10:01:50.333351+0000","last_undegraded":"2024-09-10T10:01:50.333351+0000","last_fullsized":"2024-09-10T10:01:50.333351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:19.019159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335673+0000","last_change":"2024-09-10T10:01:34.248683+0000","last_active":"2024-09-10T10:01:50.335673+0000","last_peered":"2024-09-10T10:01:50.335673+0000","last_clean":"2024-09-10T10:01:50.335673+0000","last_became_active":"2024-09-10T10:01:34.248228+0000","last_became_peered":"2024-09-10T10:01:34.248228+0000","last_unstale":"2024-09-10T10:01:50.335673+0000","last_undegraded":"2024-09-10T10:01:50.335673+0000","last_fullsized":"2024-09-10T10:01:50.335673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:00.485621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332452+0000","last_change":"2024-09-10T10:01:34.254683+0000","last_active":"2024-09-10T10:01:50.332452+0000","last_peered":"2024-09-10T10:01:50.332452+0000","last_clean":"2024-09-10T10:01:50.332452+0000","last_became_active":"2024-09-10T10:01:34.254384+0000","last_became_peered":"2024-09-10T10:01:34.254384+0000","last_unstale":"2024-09-10T10:01:50.332452+0000","last_undegraded":"2024-09-10T10:01:50.332452+0000","last_fullsized":"2024-09-10T10:01:50.332452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:35.404584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321448+0000","last_change":"2024-09-10T10:01:34.254079+0000","last_active":"2024-09-10T10:01:50.321448+0000","last_peered":"2024-09-10T10:01:50.321448+0000","last_clean":"2024-09-10T10:01:50.321448+0000","last_became_active":"2024-09-10T10:01:34.253527+0000","last_became_peered":"2024-09-10T10:01:34.253527+0000","last_unstale":"2024-09-10T10:01:50.321448+0000","last_undegraded":"2024-09-10T10:01:50.321448+0000","last_fullsized":"2024-09-10T10:01:50.321448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:31.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332948+0000","last_change":"2024-09-10T10:01:34.244840+0000","last_active":"2024-09-10T10:01:50.332948+0000","last_peered":"2024-09-10T10:01:50.332948+0000","last_clean":"2024-09-10T10:01:50.332948+0000","last_became_active":"2024-09-10T10:01:34.244482+0000","last_became_peered":"2024-09-10T10:01:34.244482+0000","last_unstale":"2024-09-10T10:01:50.332948+0000","last_undegraded":"2024-09-10T10:01:50.332948+0000","last_fullsized":"2024-09-10T10:01:50.332948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:50.666840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332939+0000","last_change":"2024-09-10T10:01:34.250789+0000","last_active":"2024-09-10T10:01:50.332939+0000","last_peered":"2024-09-10T10:01:50.332939+0000","last_clean":"2024-09-10T10:01:50.332939+0000","last_became_active":"2024-09-10T10:01:34.250445+0000","last_became_peered":"2024-09-10T10:01:34.250445+0000","last_unstale":"2024-09-10T10:01:50.332939+0000","last_undegraded":"2024-09-10T10:01:50.332939+0000","last_fullsized":"2024-09-10T10:01:50.332939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:00.389643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335618+0000","last_change":"2024-09-10T10:01:34.253139+0000","last_active":"2024-09-10T10:01:50.335618+0000","last_peered":"2024-09-10T10:01:50.335618+0000","last_clean":"2024-09-10T10:01:50.335618+0000","last_became_active":"2024-09-10T10:01:34.251801+0000","last_became_peered":"2024-09-10T10:01:34.251801+0000","last_unstale":"2024-09-10T10:01:50.335618+0000","last_undegraded":"2024-09-10T10:01:50.335618+0000","last_fullsized":"2024-09-10T10:01:50.335618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:33.492291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338268+0000","last_change":"2024-09-10T10:01:34.243669+0000","last_active":"2024-09-10T10:01:50.338268+0000","last_peered":"2024-09-10T10:01:50.338268+0000","last_clean":"2024-09-10T10:01:50.338268+0000","last_became_active":"2024-09-10T10:01:34.243291+0000","last_became_peered":"2024-09-10T10:01:34.243291+0000","last_unstale":"2024-09-10T10:01:50.338268+0000","last_undegraded":"2024-09-10T10:01:50.338268+0000","last_fullsized":"2024-09-10T10:01:50.338268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:47.157495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710729+0000","last_change":"2024-09-10T10:01:32.206317+0000","last_active":"2024-09-10T10:01:50.710729+0000","last_peered":"2024-09-10T10:01:50.710729+0000","last_clean":"2024-09-10T10:01:50.710729+0000","last_became_active":"2024-09-10T10:01:32.205412+0000","last_became_peered":"2024-09-10T10:01:32.205412+0000","last_unstale":"2024-09-10T10:01:50.710729+0000","last_undegraded":"2024-09-10T10:01:50.710729+0000","last_fullsized":"2024-09-10T10:01:50.710729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:07.489923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710763+0000","last_change":"2024-09-10T10:01:34.254783+0000","last_active":"2024-09-10T10:01:50.710763+0000","last_peered":"2024-09-10T10:01:50.710763+0000","last_clean":"2024-09-10T10:01:50.710763+0000","last_became_active":"2024-09-10T10:01:34.254465+0000","last_became_peered":"2024-09-10T10:01:34.254465+0000","last_unstale":"2024-09-10T10:01:50.710763+0000","last_undegraded":"2024-09-10T10:01:50.710763+0000","last_fullsized":"2024-09-10T10:01:50.710763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:57.983265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320605+0000","last_change":"2024-09-10T10:01:32.204551+0000","last_active":"2024-09-10T10:01:50.320605+0000","last_peered":"2024-09-10T10:01:50.320605+0000","last_clean":"2024-09-10T10:01:50.320605+0000","last_became_active":"2024-09-10T10:01:32.204214+0000","last_became_peered":"2024-09-10T10:01:32.204214+0000","last_unstale":"2024-09-10T10:01:50.320605+0000","last_undegraded":"2024-09-10T10:01:50.320605+0000","last_fullsized":"2024-09-10T10:01:50.320605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:50.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332156+0000","last_change":"2024-09-10T10:01:32.205907+0000","last_active":"2024-09-10T10:01:50.332156+0000","last_peered":"2024-09-10T10:01:50.332156+0000","last_clean":"2024-09-10T10:01:50.332156+0000","last_became_active":"2024-09-10T10:01:32.205523+0000","last_became_peered":"2024-09-10T10:01:32.205523+0000","last_unstale":"2024-09-10T10:01:50.332156+0000","last_undegraded":"2024-09-10T10:01:50.332156+0000","last_fullsized":"2024-09-10T10:01:50.332156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:14.540234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319983+0000","last_change":"2024-09-10T10:01:34.249826+0000","last_active":"2024-09-10T10:01:50.319983+0000","last_peered":"2024-09-10T10:01:50.319983+0000","last_clean":"2024-09-10T10:01:50.319983+0000","last_became_active":"2024-09-10T10:01:34.248940+0000","last_became_peered":"2024-09-10T10:01:34.248940+0000","last_unstale":"2024-09-10T10:01:50.319983+0000","last_undegraded":"2024-09-10T10:01:50.319983+0000","last_fullsized":"2024-09-10T10:01:50.319983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:49.125909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710836+0000","last_change":"2024-09-10T10:01:32.205559+0000","last_active":"2024-09-10T10:01:50.710836+0000","last_peered":"2024-09-10T10:01:50.710836+0000","last_clean":"2024-09-10T10:01:50.710836+0000","last_became_active":"2024-09-10T10:01:32.205251+0000","last_became_peered":"2024-09-10T10:01:32.205251+0000","last_unstale":"2024-09-10T10:01:50.710836+0000","last_undegraded":"2024-09-10T10:01:50.710836+0000","last_fullsized":"2024-09-10T10:01:50.710836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:11.442234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317525+0000","last_change":"2024-09-10T10:01:34.245852+0000","last_active":"2024-09-10T10:01:50.317525+0000","last_peered":"2024-09-10T10:01:50.317525+0000","last_clean":"2024-09-10T10:01:50.317525+0000","last_became_active":"2024-09-10T10:01:34.245487+0000","last_became_peered":"2024-09-10T10:01:34.245487+0000","last_unstale":"2024-09-10T10:01:50.317525+0000","last_undegraded":"2024-09-10T10:01:50.317525+0000","last_fullsized":"2024-09-10T10:01:50.317525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:01.995443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320839+0000","last_change":"2024-09-10T10:01:32.202754+0000","last_active":"2024-09-10T10:01:50.320839+0000","last_peered":"2024-09-10T10:01:50.320839+0000","last_clean":"2024-09-10T10:01:50.320839+0000","last_became_active":"2024-09-10T10:01:32.202360+0000","last_became_peered":"2024-09-10T10:01:32.202360+0000","last_unstale":"2024-09-10T10:01:50.320839+0000","last_undegraded":"2024-09-10T10:01:50.320839+0000","last_fullsized":"2024-09-10T10:01:50.320839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:04.864276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318627+0000","last_change":"2024-09-10T10:01:34.245670+0000","last_active":"2024-09-10T10:01:50.318627+0000","last_peered":"2024-09-10T10:01:50.318627+0000","last_clean":"2024-09-10T10:01:50.318627+0000","last_became_active":"2024-09-10T10:01:34.244922+0000","last_became_peered":"2024-09-10T10:01:34.244922+0000","last_unstale":"2024-09-10T10:01:50.318627+0000","last_undegraded":"2024-09-10T10:01:50.318627+0000","last_fullsized":"2024-09-10T10:01:50.318627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:05.442144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338108+0000","last_change":"2024-09-10T10:01:32.204658+0000","last_active":"2024-09-10T10:01:50.338108+0000","last_peered":"2024-09-10T10:01:50.338108+0000","last_clean":"2024-09-10T10:01:50.338108+0000","last_became_active":"2024-09-10T10:01:32.204310+0000","last_became_peered":"2024-09-10T10:01:32.204310+0000","last_unstale":"2024-09-10T10:01:50.338108+0000","last_undegraded":"2024-09-10T10:01:50.338108+0000","last_fullsized":"2024-09-10T10:01:50.338108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:16.446977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.712239+0000","last_change":"2024-09-10T10:01:50.712239+0000","last_active":"2024-09-10T10:01:50.712239+0000","last_peered":"2024-09-10T10:01:50.712239+0000","last_clean":"2024-09-10T10:01:50.712239+0000","last_became_active":"2024-09-10T10:01:50.711826+0000","last_became_peered":"2024-09-10T10:01:50.711826+0000","last_unstale":"2024-09-10T10:01:50.712239+0000","last_undegraded":"2024-09-10T10:01:50.712239+0000","last_fullsized":"2024-09-10T10:01:50.712239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:54.670359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710657+0000","last_change":"2024-09-10T10:01:32.530649+0000","last_active":"2024-09-10T10:01:50.710657+0000","last_peered":"2024-09-10T10:01:50.710657+0000","last_clean":"2024-09-10T10:01:50.710657+0000","last_became_active":"2024-09-10T10:01:32.530157+0000","last_became_peered":"2024-09-10T10:01:32.530157+0000","last_unstale":"2024-09-10T10:01:50.710657+0000","last_undegraded":"2024-09-10T10:01:50.710657+0000","last_fullsized":"2024-09-10T10:01:50.710657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:30.235163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339461+0000","last_change":"2024-09-10T10:01:50.339461+0000","last_active":"2024-09-10T10:01:50.339461+0000","last_peered":"2024-09-10T10:01:50.339461+0000","last_clean":"2024-09-10T10:01:50.339461+0000","last_became_active":"2024-09-10T10:01:50.339158+0000","last_became_peered":"2024-09-10T10:01:50.339158+0000","last_unstale":"2024-09-10T10:01:50.339461+0000","last_undegraded":"2024-09-10T10:01:50.339461+0000","last_fullsized":"2024-09-10T10:01:50.339461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:11.733302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'27","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.357860+0000","last_change":"2024-09-10T10:01:32.205933+0000","last_active":"2024-09-10T10:02:31.357860+0000","last_peered":"2024-09-10T10:02:31.357860+0000","last_clean":"2024-09-10T10:02:31.357860+0000","last_became_active":"2024-09-10T10:01:32.205592+0000","last_became_peered":"2024-09-10T10:01:32.205592+0000","last_unstale":"2024-09-10T10:02:31.357860+0000","last_undegraded":"2024-09-10T10:02:31.357860+0000","last_fullsized":"2024-09-10T10:02:31.357860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:29.107388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62819,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":109,"num_read_kb":115,"num_write":27,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338800+0000","last_change":"2024-09-10T10:01:34.249720+0000","last_active":"2024-09-10T10:01:50.338800+0000","last_peered":"2024-09-10T10:01:50.338800+0000","last_clean":"2024-09-10T10:01:50.338800+0000","last_became_active":"2024-09-10T10:01:34.249285+0000","last_became_peered":"2024-09-10T10:01:34.249285+0000","last_unstale":"2024-09-10T10:01:50.338800+0000","last_undegraded":"2024-09-10T10:01:50.338800+0000","last_fullsized":"2024-09-10T10:01:50.338800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:30.787344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333124+0000","last_change":"2024-09-10T10:01:32.202622+0000","last_active":"2024-09-10T10:01:50.333124+0000","last_peered":"2024-09-10T10:01:50.333124+0000","last_clean":"2024-09-10T10:01:50.333124+0000","last_became_active":"2024-09-10T10:01:32.200870+0000","last_became_peered":"2024-09-10T10:01:32.200870+0000","last_unstale":"2024-09-10T10:01:50.333124+0000","last_undegraded":"2024-09-10T10:01:50.333124+0000","last_fullsized":"2024-09-10T10:01:50.333124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:48.562971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332189+0000","last_change":"2024-09-10T10:01:34.252622+0000","last_active":"2024-09-10T10:01:50.332189+0000","last_peered":"2024-09-10T10:01:50.332189+0000","last_clean":"2024-09-10T10:01:50.332189+0000","last_became_active":"2024-09-10T10:01:34.252271+0000","last_became_peered":"2024-09-10T10:01:34.252271+0000","last_unstale":"2024-09-10T10:01:50.332189+0000","last_undegraded":"2024-09-10T10:01:50.332189+0000","last_fullsized":"2024-09-10T10:01:50.332189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:03.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710592+0000","last_change":"2024-09-10T10:01:32.206133+0000","last_active":"2024-09-10T10:01:50.710592+0000","last_peered":"2024-09-10T10:01:50.710592+0000","last_clean":"2024-09-10T10:01:50.710592+0000","last_became_active":"2024-09-10T10:01:32.205747+0000","last_became_peered":"2024-09-10T10:01:32.205747+0000","last_unstale":"2024-09-10T10:01:50.710592+0000","last_undegraded":"2024-09-10T10:01:50.710592+0000","last_fullsized":"2024-09-10T10:01:50.710592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:30.515614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337830+0000","last_change":"2024-09-10T10:01:34.257702+0000","last_active":"2024-09-10T10:01:50.337830+0000","last_peered":"2024-09-10T10:01:50.337830+0000","last_clean":"2024-09-10T10:01:50.337830+0000","last_became_active":"2024-09-10T10:01:34.257366+0000","last_became_peered":"2024-09-10T10:01:34.257366+0000","last_unstale":"2024-09-10T10:01:50.337830+0000","last_undegraded":"2024-09-10T10:01:50.337830+0000","last_fullsized":"2024-09-10T10:01:50.337830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:51.772522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838725+0000","last_change":"2024-09-10T10:01:32.205527+0000","last_active":"2024-09-10T10:01:50.838725+0000","last_peered":"2024-09-10T10:01:50.838725+0000","last_clean":"2024-09-10T10:01:50.838725+0000","last_became_active":"2024-09-10T10:01:32.205162+0000","last_became_peered":"2024-09-10T10:01:32.205162+0000","last_unstale":"2024-09-10T10:01:50.838725+0000","last_undegraded":"2024-09-10T10:01:50.838725+0000","last_fullsized":"2024-09-10T10:01:50.838725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:28.149331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337435+0000","last_change":"2024-09-10T10:01:34.250691+0000","last_active":"2024-09-10T10:01:50.337435+0000","last_peered":"2024-09-10T10:01:50.337435+0000","last_clean":"2024-09-10T10:01:50.337435+0000","last_became_active":"2024-09-10T10:01:34.249303+0000","last_became_peered":"2024-09-10T10:01:34.249303+0000","last_unstale":"2024-09-10T10:01:50.337435+0000","last_undegraded":"2024-09-10T10:01:50.337435+0000","last_fullsized":"2024-09-10T10:01:50.337435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:47.413567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338071+0000","last_change":"2024-09-10T10:01:32.530043+0000","last_active":"2024-09-10T10:01:50.338071+0000","last_peered":"2024-09-10T10:01:50.338071+0000","last_clean":"2024-09-10T10:01:50.338071+0000","last_became_active":"2024-09-10T10:01:32.529502+0000","last_became_peered":"2024-09-10T10:01:32.529502+0000","last_unstale":"2024-09-10T10:01:50.338071+0000","last_undegraded":"2024-09-10T10:01:50.338071+0000","last_fullsized":"2024-09-10T10:01:50.338071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:26.133771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710304+0000","last_change":"2024-09-10T10:01:34.254722+0000","last_active":"2024-09-10T10:01:50.710304+0000","last_peered":"2024-09-10T10:01:50.710304+0000","last_clean":"2024-09-10T10:01:50.710304+0000","last_became_active":"2024-09-10T10:01:34.254390+0000","last_became_peered":"2024-09-10T10:01:34.254390+0000","last_unstale":"2024-09-10T10:01:50.710304+0000","last_undegraded":"2024-09-10T10:01:50.710304+0000","last_fullsized":"2024-09-10T10:01:50.710304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:37.261256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339061+0000","last_change":"2024-09-10T10:01:32.200744+0000","last_active":"2024-09-10T10:01:50.339061+0000","last_peered":"2024-09-10T10:01:50.339061+0000","last_clean":"2024-09-10T10:01:50.339061+0000","last_became_active":"2024-09-10T10:01:32.199906+0000","last_became_peered":"2024-09-10T10:01:32.199906+0000","last_unstale":"2024-09-10T10:01:50.339061+0000","last_undegraded":"2024-09-10T10:01:50.339061+0000","last_fullsized":"2024-09-10T10:01:50.339061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:17.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.840135+0000","last_change":"2024-09-10T10:01:50.840135+0000","last_active":"2024-09-10T10:01:50.840135+0000","last_peered":"2024-09-10T10:01:50.840135+0000","last_clean":"2024-09-10T10:01:50.840135+0000","last_became_active":"2024-09-10T10:01:50.839693+0000","last_became_peered":"2024-09-10T10:01:50.839693+0000","last_unstale":"2024-09-10T10:01:50.840135+0000","last_undegraded":"2024-09-10T10:01:50.840135+0000","last_fullsized":"2024-09-10T10:01:50.840135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:01.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332542+0000","last_change":"2024-09-10T10:01:32.200383+0000","last_active":"2024-09-10T10:01:50.332542+0000","last_peered":"2024-09-10T10:01:50.332542+0000","last_clean":"2024-09-10T10:01:50.332542+0000","last_became_active":"2024-09-10T10:01:32.199926+0000","last_became_peered":"2024-09-10T10:01:32.199926+0000","last_unstale":"2024-09-10T10:01:50.332542+0000","last_undegraded":"2024-09-10T10:01:50.332542+0000","last_fullsized":"2024-09-10T10:01:50.332542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:29.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338221+0000","last_change":"2024-09-10T10:01:34.250077+0000","last_active":"2024-09-10T10:01:50.338221+0000","last_peered":"2024-09-10T10:01:50.338221+0000","last_clean":"2024-09-10T10:01:50.338221+0000","last_became_active":"2024-09-10T10:01:34.249697+0000","last_became_peered":"2024-09-10T10:01:34.249697+0000","last_unstale":"2024-09-10T10:01:50.338221+0000","last_undegraded":"2024-09-10T10:01:50.338221+0000","last_fullsized":"2024-09-10T10:01:50.338221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:25.068115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710730+0000","last_change":"2024-09-10T10:01:32.200235+0000","last_active":"2024-09-10T10:01:50.710730+0000","last_peered":"2024-09-10T10:01:50.710730+0000","last_clean":"2024-09-10T10:01:50.710730+0000","last_became_active":"2024-09-10T10:01:32.199920+0000","last_became_peered":"2024-09-10T10:01:32.199920+0000","last_unstale":"2024-09-10T10:01:50.710730+0000","last_undegraded":"2024-09-10T10:01:50.710730+0000","last_fullsized":"2024-09-10T10:01:50.710730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:10.980054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338054+0000","last_change":"2024-09-10T10:01:34.256937+0000","last_active":"2024-09-10T10:01:50.338054+0000","last_peered":"2024-09-10T10:01:50.338054+0000","last_clean":"2024-09-10T10:01:50.338054+0000","last_became_active":"2024-09-10T10:01:34.256376+0000","last_became_peered":"2024-09-10T10:01:34.256376+0000","last_unstale":"2024-09-10T10:01:50.338054+0000","last_undegraded":"2024-09-10T10:01:50.338054+0000","last_fullsized":"2024-09-10T10:01:50.338054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:07.386683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333436+0000","last_change":"2024-09-10T10:01:32.530794+0000","last_active":"2024-09-10T10:01:50.333436+0000","last_peered":"2024-09-10T10:01:50.333436+0000","last_clean":"2024-09-10T10:01:50.333436+0000","last_became_active":"2024-09-10T10:01:32.530335+0000","last_became_peered":"2024-09-10T10:01:32.530335+0000","last_unstale":"2024-09-10T10:01:50.333436+0000","last_undegraded":"2024-09-10T10:01:50.333436+0000","last_fullsized":"2024-09-10T10:01:50.333436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:40.883029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338123+0000","last_change":"2024-09-10T10:01:34.250228+0000","last_active":"2024-09-10T10:01:50.338123+0000","last_peered":"2024-09-10T10:01:50.338123+0000","last_clean":"2024-09-10T10:01:50.338123+0000","last_became_active":"2024-09-10T10:01:34.249901+0000","last_became_peered":"2024-09-10T10:01:34.249901+0000","last_unstale":"2024-09-10T10:01:50.338123+0000","last_undegraded":"2024-09-10T10:01:50.338123+0000","last_fullsized":"2024-09-10T10:01:50.338123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:10.710185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337584+0000","last_change":"2024-09-10T10:00:23.346149+0000","last_active":"2024-09-10T10:01:50.337584+0000","last_peered":"2024-09-10T10:01:50.337584+0000","last_clean":"2024-09-10T10:01:50.337584+0000","last_became_active":"2024-09-10T10:00:23.345608+0000","last_became_peered":"2024-09-10T10:00:23.345608+0000","last_unstale":"2024-09-10T10:01:50.337584+0000","last_undegraded":"2024-09-10T10:01:50.337584+0000","last_fullsized":"2024-09-10T10:01:50.337584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_clean_scrub_stamp":"2024-09-10T09:57:03.194756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:14.092838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838755+0000","last_change":"2024-09-10T10:01:32.200726+0000","last_active":"2024-09-10T10:01:50.838755+0000","last_peered":"2024-09-10T10:01:50.838755+0000","last_clean":"2024-09-10T10:01:50.838755+0000","last_became_active":"2024-09-10T10:01:32.200163+0000","last_became_peered":"2024-09-10T10:01:32.200163+0000","last_unstale":"2024-09-10T10:01:50.838755+0000","last_undegraded":"2024-09-10T10:01:50.838755+0000","last_fullsized":"2024-09-10T10:01:50.838755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:05.300476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320505+0000","last_change":"2024-09-10T10:01:34.243258+0000","last_active":"2024-09-10T10:01:50.320505+0000","last_peered":"2024-09-10T10:01:50.320505+0000","last_clean":"2024-09-10T10:01:50.320505+0000","last_became_active":"2024-09-10T10:01:34.242060+0000","last_became_peered":"2024-09-10T10:01:34.242060+0000","last_unstale":"2024-09-10T10:01:50.320505+0000","last_undegraded":"2024-09-10T10:01:50.320505+0000","last_fullsized":"2024-09-10T10:01:50.320505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:02.910538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337582+0000","last_change":"2024-09-10T10:01:32.205174+0000","last_active":"2024-09-10T10:01:50.337582+0000","last_peered":"2024-09-10T10:01:50.337582+0000","last_clean":"2024-09-10T10:01:50.337582+0000","last_became_active":"2024-09-10T10:01:32.204121+0000","last_became_peered":"2024-09-10T10:01:32.204121+0000","last_unstale":"2024-09-10T10:01:50.337582+0000","last_undegraded":"2024-09-10T10:01:50.337582+0000","last_fullsized":"2024-09-10T10:01:50.337582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:42.533591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332204+0000","last_change":"2024-09-10T10:01:34.254895+0000","last_active":"2024-09-10T10:01:50.332204+0000","last_peered":"2024-09-10T10:01:50.332204+0000","last_clean":"2024-09-10T10:01:50.332204+0000","last_became_active":"2024-09-10T10:01:34.254572+0000","last_became_peered":"2024-09-10T10:01:34.254572+0000","last_unstale":"2024-09-10T10:01:50.332204+0000","last_undegraded":"2024-09-10T10:01:50.332204+0000","last_fullsized":"2024-09-10T10:01:50.332204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:44.194941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332304+0000","last_change":"2024-09-10T10:01:32.530093+0000","last_active":"2024-09-10T10:01:50.332304+0000","last_peered":"2024-09-10T10:01:50.332304+0000","last_clean":"2024-09-10T10:01:50.332304+0000","last_became_active":"2024-09-10T10:01:32.529756+0000","last_became_peered":"2024-09-10T10:01:32.529756+0000","last_unstale":"2024-09-10T10:01:50.332304+0000","last_undegraded":"2024-09-10T10:01:50.332304+0000","last_fullsized":"2024-09-10T10:01:50.332304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.520549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710348+0000","last_change":"2024-09-10T10:01:34.255210+0000","last_active":"2024-09-10T10:01:50.710348+0000","last_peered":"2024-09-10T10:01:50.710348+0000","last_clean":"2024-09-10T10:01:50.710348+0000","last_became_active":"2024-09-10T10:01:34.254861+0000","last_became_peered":"2024-09-10T10:01:34.254861+0000","last_unstale":"2024-09-10T10:01:50.710348+0000","last_undegraded":"2024-09-10T10:01:50.710348+0000","last_fullsized":"2024-09-10T10:01:50.710348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:10.332758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337852+0000","last_change":"2024-09-10T10:01:32.204622+0000","last_active":"2024-09-10T10:01:50.337852+0000","last_peered":"2024-09-10T10:01:50.337852+0000","last_clean":"2024-09-10T10:01:50.337852+0000","last_became_active":"2024-09-10T10:01:32.204236+0000","last_became_peered":"2024-09-10T10:01:32.204236+0000","last_unstale":"2024-09-10T10:01:50.337852+0000","last_undegraded":"2024-09-10T10:01:50.337852+0000","last_fullsized":"2024-09-10T10:01:50.337852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:59.402854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320679+0000","last_change":"2024-09-10T10:01:34.244790+0000","last_active":"2024-09-10T10:01:50.320679+0000","last_peered":"2024-09-10T10:01:50.320679+0000","last_clean":"2024-09-10T10:01:50.320679+0000","last_became_active":"2024-09-10T10:01:34.244443+0000","last_became_peered":"2024-09-10T10:01:34.244443+0000","last_unstale":"2024-09-10T10:01:50.320679+0000","last_undegraded":"2024-09-10T10:01:50.320679+0000","last_fullsized":"2024-09-10T10:01:50.320679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:55.913475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.529774+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.529410+0000","last_became_peered":"2024-09-10T10:01:32.529410+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:01.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335478+0000","last_change":"2024-09-10T10:01:50.335478+0000","last_active":"2024-09-10T10:01:50.335478+0000","last_peered":"2024-09-10T10:01:50.335478+0000","last_clean":"2024-09-10T10:01:50.335478+0000","last_became_active":"2024-09-10T10:01:50.332726+0000","last_became_peered":"2024-09-10T10:01:50.332726+0000","last_unstale":"2024-09-10T10:01:50.335478+0000","last_undegraded":"2024-09-10T10:01:50.335478+0000","last_fullsized":"2024-09-10T10:01:50.335478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:04:10.726066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338880+0000","last_change":"2024-09-10T10:01:32.530401+0000","last_active":"2024-09-10T10:01:50.338880+0000","last_peered":"2024-09-10T10:01:50.338880+0000","last_clean":"2024-09-10T10:01:50.338880+0000","last_became_active":"2024-09-10T10:01:32.529928+0000","last_became_peered":"2024-09-10T10:01:32.529928+0000","last_unstale":"2024-09-10T10:01:50.338880+0000","last_undegraded":"2024-09-10T10:01:50.338880+0000","last_fullsized":"2024-09-10T10:01:50.338880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:18.978923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332474+0000","last_change":"2024-09-10T10:01:34.245777+0000","last_active":"2024-09-10T10:01:50.332474+0000","last_peered":"2024-09-10T10:01:50.332474+0000","last_clean":"2024-09-10T10:01:50.332474+0000","last_became_active":"2024-09-10T10:01:34.245431+0000","last_became_peered":"2024-09-10T10:01:34.245431+0000","last_unstale":"2024-09-10T10:01:50.332474+0000","last_undegraded":"2024-09-10T10:01:50.332474+0000","last_fullsized":"2024-09-10T10:01:50.332474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:17.754117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.246544+0000","last_change":"2024-09-10T10:01:32.201916+0000","last_active":"2024-09-10T10:01:56.246544+0000","last_peered":"2024-09-10T10:01:56.246544+0000","last_clean":"2024-09-10T10:01:56.246544+0000","last_became_active":"2024-09-10T10:01:32.201540+0000","last_became_peered":"2024-09-10T10:01:32.201540+0000","last_unstale":"2024-09-10T10:01:56.246544+0000","last_undegraded":"2024-09-10T10:01:56.246544+0000","last_fullsized":"2024-09-10T10:01:56.246544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:41.862673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320509+0000","last_change":"2024-09-10T10:01:34.251334+0000","last_active":"2024-09-10T10:01:50.320509+0000","last_peered":"2024-09-10T10:01:50.320509+0000","last_clean":"2024-09-10T10:01:50.320509+0000","last_became_active":"2024-09-10T10:01:34.251032+0000","last_became_peered":"2024-09-10T10:01:34.251032+0000","last_unstale":"2024-09-10T10:01:50.320509+0000","last_undegraded":"2024-09-10T10:01:50.320509+0000","last_fullsized":"2024-09-10T10:01:50.320509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:26.437259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.839313+0000","last_change":"2024-09-10T10:01:32.255074+0000","last_active":"2024-09-10T10:01:50.839313+0000","last_peered":"2024-09-10T10:01:50.839313+0000","last_clean":"2024-09-10T10:01:50.839313+0000","last_became_active":"2024-09-10T10:01:32.254637+0000","last_became_peered":"2024-09-10T10:01:32.254637+0000","last_unstale":"2024-09-10T10:01:50.839313+0000","last_undegraded":"2024-09-10T10:01:50.839313+0000","last_fullsized":"2024-09-10T10:01:50.839313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:35.392816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321297+0000","last_change":"2024-09-10T10:01:34.253770+0000","last_active":"2024-09-10T10:01:50.321297+0000","last_peered":"2024-09-10T10:01:50.321297+0000","last_clean":"2024-09-10T10:01:50.321297+0000","last_became_active":"2024-09-10T10:01:34.253395+0000","last_became_peered":"2024-09-10T10:01:34.253395+0000","last_unstale":"2024-09-10T10:01:50.321297+0000","last_undegraded":"2024-09-10T10:01:50.321297+0000","last_fullsized":"2024-09-10T10:01:50.321297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:55.264734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337074+0000","last_change":"2024-09-10T10:01:50.337074+0000","last_active":"2024-09-10T10:01:50.337074+0000","last_peered":"2024-09-10T10:01:50.337074+0000","last_clean":"2024-09-10T10:01:50.337074+0000","last_became_active":"2024-09-10T10:01:50.336661+0000","last_became_peered":"2024-09-10T10:01:50.336661+0000","last_unstale":"2024-09-10T10:01:50.337074+0000","last_undegraded":"2024-09-10T10:01:50.337074+0000","last_fullsized":"2024-09-10T10:01:50.337074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:47.700516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338318+0000","last_change":"2024-09-10T10:01:34.249342+0000","last_active":"2024-09-10T10:01:50.338318+0000","last_peered":"2024-09-10T10:01:50.338318+0000","last_clean":"2024-09-10T10:01:50.338318+0000","last_became_active":"2024-09-10T10:01:34.245289+0000","last_became_peered":"2024-09-10T10:01:34.245289+0000","last_unstale":"2024-09-10T10:01:50.338318+0000","last_undegraded":"2024-09-10T10:01:50.338318+0000","last_fullsized":"2024-09-10T10:01:50.338318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:34.068515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320070+0000","last_change":"2024-09-10T10:01:32.199112+0000","last_active":"2024-09-10T10:01:50.320070+0000","last_peered":"2024-09-10T10:01:50.320070+0000","last_clean":"2024-09-10T10:01:50.320070+0000","last_became_active":"2024-09-10T10:01:32.198627+0000","last_became_peered":"2024-09-10T10:01:32.198627+0000","last_unstale":"2024-09-10T10:01:50.320070+0000","last_undegraded":"2024-09-10T10:01:50.320070+0000","last_fullsized":"2024-09-10T10:01:50.320070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:37.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335789+0000","last_change":"2024-09-10T10:01:34.253537+0000","last_active":"2024-09-10T10:01:50.335789+0000","last_peered":"2024-09-10T10:01:50.335789+0000","last_clean":"2024-09-10T10:01:50.335789+0000","last_became_active":"2024-09-10T10:01:34.253060+0000","last_became_peered":"2024-09-10T10:01:34.253060+0000","last_unstale":"2024-09-10T10:01:50.335789+0000","last_undegraded":"2024-09-10T10:01:50.335789+0000","last_fullsized":"2024-09-10T10:01:50.335789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.411289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333010+0000","last_change":"2024-09-10T10:01:32.202595+0000","last_active":"2024-09-10T10:01:50.333010+0000","last_peered":"2024-09-10T10:01:50.333010+0000","last_clean":"2024-09-10T10:01:50.333010+0000","last_became_active":"2024-09-10T10:01:32.201796+0000","last_became_peered":"2024-09-10T10:01:32.201796+0000","last_unstale":"2024-09-10T10:01:50.333010+0000","last_undegraded":"2024-09-10T10:01:50.333010+0000","last_fullsized":"2024-09-10T10:01:50.333010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:54.778442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320621+0000","last_change":"2024-09-10T10:01:34.244734+0000","last_active":"2024-09-10T10:01:50.320621+0000","last_peered":"2024-09-10T10:01:50.320621+0000","last_clean":"2024-09-10T10:01:50.320621+0000","last_became_active":"2024-09-10T10:01:34.244396+0000","last_became_peered":"2024-09-10T10:01:34.244396+0000","last_unstale":"2024-09-10T10:01:50.320621+0000","last_undegraded":"2024-09-10T10:01:50.320621+0000","last_fullsized":"2024-09-10T10:01:50.320621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:02.266670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338190+0000","last_change":"2024-09-10T10:01:32.207322+0000","last_active":"2024-09-10T10:01:50.338190+0000","last_peered":"2024-09-10T10:01:50.338190+0000","last_clean":"2024-09-10T10:01:50.338190+0000","last_became_active":"2024-09-10T10:01:32.206936+0000","last_became_peered":"2024-09-10T10:01:32.206936+0000","last_unstale":"2024-09-10T10:01:50.338190+0000","last_undegraded":"2024-09-10T10:01:50.338190+0000","last_fullsized":"2024-09-10T10:01:50.338190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:17.194749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337740+0000","last_change":"2024-09-10T10:01:34.255626+0000","last_active":"2024-09-10T10:01:50.337740+0000","last_peered":"2024-09-10T10:01:50.337740+0000","last_clean":"2024-09-10T10:01:50.337740+0000","last_became_active":"2024-09-10T10:01:34.255333+0000","last_became_peered":"2024-09-10T10:01:34.255333+0000","last_unstale":"2024-09-10T10:01:50.337740+0000","last_undegraded":"2024-09-10T10:01:50.337740+0000","last_fullsized":"2024-09-10T10:01:50.337740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:26.287576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710505+0000","last_change":"2024-09-10T10:01:34.257209+0000","last_active":"2024-09-10T10:01:50.710505+0000","last_peered":"2024-09-10T10:01:50.710505+0000","last_clean":"2024-09-10T10:01:50.710505+0000","last_became_active":"2024-09-10T10:01:34.256891+0000","last_became_peered":"2024-09-10T10:01:34.256891+0000","last_unstale":"2024-09-10T10:01:50.710505+0000","last_undegraded":"2024-09-10T10:01:50.710505+0000","last_fullsized":"2024-09-10T10:01:50.710505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:58.178876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338664+0000","last_change":"2024-09-10T10:01:32.200462+0000","last_active":"2024-09-10T10:01:50.338664+0000","last_peered":"2024-09-10T10:01:50.338664+0000","last_clean":"2024-09-10T10:01:50.338664+0000","last_became_active":"2024-09-10T10:01:32.198970+0000","last_became_peered":"2024-09-10T10:01:32.198970+0000","last_unstale":"2024-09-10T10:01:50.338664+0000","last_undegraded":"2024-09-10T10:01:50.338664+0000","last_fullsized":"2024-09-10T10:01:50.338664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:00.980894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318326+0000","last_change":"2024-09-10T10:01:34.246984+0000","last_active":"2024-09-10T10:01:50.318326+0000","last_peered":"2024-09-10T10:01:50.318326+0000","last_clean":"2024-09-10T10:01:50.318326+0000","last_became_active":"2024-09-10T10:01:34.246244+0000","last_became_peered":"2024-09-10T10:01:34.246244+0000","last_unstale":"2024-09-10T10:01:50.318326+0000","last_undegraded":"2024-09-10T10:01:50.318326+0000","last_fullsized":"2024-09-10T10:01:50.318326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:47.656959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.342048+0000","last_change":"2024-09-10T10:01:50.341594+0000","last_active":"2024-09-10T10:01:50.342048+0000","last_peered":"2024-09-10T10:01:50.342048+0000","last_clean":"2024-09-10T10:01:50.342048+0000","last_became_active":"2024-09-10T10:01:50.338152+0000","last_became_peered":"2024-09-10T10:01:50.338152+0000","last_unstale":"2024-09-10T10:01:50.342048+0000","last_undegraded":"2024-09-10T10:01:50.342048+0000","last_fullsized":"2024-09-10T10:01:50.342048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:25.484321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320803+0000","last_change":"2024-09-10T10:01:34.246396+0000","last_active":"2024-09-10T10:01:50.320803+0000","last_peered":"2024-09-10T10:01:50.320803+0000","last_clean":"2024-09-10T10:01:50.320803+0000","last_became_active":"2024-09-10T10:01:34.245956+0000","last_became_peered":"2024-09-10T10:01:34.245956+0000","last_unstale":"2024-09-10T10:01:50.320803+0000","last_undegraded":"2024-09-10T10:01:50.320803+0000","last_fullsized":"2024-09-10T10:01:50.320803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:53.440984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320437+0000","last_change":"2024-09-10T10:01:32.529907+0000","last_active":"2024-09-10T10:01:50.320437+0000","last_peered":"2024-09-10T10:01:50.320437+0000","last_clean":"2024-09-10T10:01:50.320437+0000","last_became_active":"2024-09-10T10:01:32.529353+0000","last_became_peered":"2024-09-10T10:01:32.529353+0000","last_unstale":"2024-09-10T10:01:50.320437+0000","last_undegraded":"2024-09-10T10:01:50.320437+0000","last_fullsized":"2024-09-10T10:01:50.320437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:45.562270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320201+0000","last_change":"2024-09-10T10:01:34.243623+0000","last_active":"2024-09-10T10:01:50.320201+0000","last_peered":"2024-09-10T10:01:50.320201+0000","last_clean":"2024-09-10T10:01:50.320201+0000","last_became_active":"2024-09-10T10:01:34.241625+0000","last_became_peered":"2024-09-10T10:01:34.241625+0000","last_unstale":"2024-09-10T10:01:50.320201+0000","last_undegraded":"2024-09-10T10:01:50.320201+0000","last_fullsized":"2024-09-10T10:01:50.320201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:25.112837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.205514+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.205127+0000","last_became_peered":"2024-09-10T10:01:32.205127+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.207240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63617,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":115,"num_read_kb":121,"num_write":83,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":190851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":437776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1091274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939682,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40968,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704184,"statfs":{"total":95995035648,"available":95953084416,"internally_reserved":0,"allocated":1495040,"data_stored":1028843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1503232,"data_stored":1028443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":437798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":437997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":500629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":500595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:36.707 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T10:02:36.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:36 smithi162 ceph-mon[34963]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:36.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:36.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:36 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:36.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:36 smithi047 ceph-mon[35244]: pgmap v306: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:36.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:36.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:36 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:36.983 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.465+0000 7f4a47aee640 1 -- 172.21.15.45:0/2431748192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a40101280 msgr2=0x7f4a4010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.465+0000 7f4a47aee640 1 --2- 172.21.15.45:0/2431748192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a40101280 0x7f4a4010df40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009900 tx=0x7f4a3402a410 comp rx=0 tx=0).stop 2024-09-10T10:02:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.465+0000 7f4a47aee640 1 -- 172.21.15.45:0/2431748192 shutdown_connections 2024-09-10T10:02:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.465+0000 7f4a47aee640 1 --2- 172.21.15.45:0/2431748192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a4010e480 0x7f4a40110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.465+0000 7f4a47aee640 1 --2- 172.21.15.45:0/2431748192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a40101280 0x7f4a4010df40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.465+0000 7f4a47aee640 1 --2- 172.21.15.45:0/2431748192 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a401008d0 0x7f4a40100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.465+0000 7f4a47aee640 1 -- 172.21.15.45:0/2431748192 >> 172.21.15.45:0/2431748192 conn(0x7f4a400fc7a0 msgr2=0x7f4a400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.466+0000 7f4a47aee640 1 -- 172.21.15.45:0/2431748192 shutdown_connections 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.466+0000 7f4a47aee640 1 -- 172.21.15.45:0/2431748192 wait complete. 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.466+0000 7f4a47aee640 1 Processor -- start 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.466+0000 7f4a47aee640 1 -- start start 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.466+0000 7f4a47aee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a401008d0 0x7f4a401a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.466+0000 7f4a47aee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a40101280 0x7f4a401a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.466+0000 7f4a47aee640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a4010e480 0x7f4a4019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a47aee640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40114200 con 0x7f4a401008d0 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a47aee640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40114080 con 0x7f4a40101280 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a47aee640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40114380 con 0x7f4a4010e480 2024-09-10T10:02:37.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a401008d0 0x7f4a401a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a401008d0 0x7f4a401a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59124/0 (socket says 172.21.15.45:59124) 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 -- 172.21.15.45:0/3296118990 learned_addr learned my addr 172.21.15.45:0/3296118990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4608b640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a4010e480 0x7f4a4019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a45089640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a40101280 0x7f4a401a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 -- 172.21.15.45:0/3296118990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a4010e480 msgr2=0x7f4a4019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a4010e480 0x7f4a4019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 -- 172.21.15.45:0/3296118990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a40101280 msgr2=0x7f4a401a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a40101280 0x7f4a401a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 -- 172.21.15.45:0/3296118990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4019acc0 con 0x7f4a401008d0 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4608b640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a4010e480 0x7f4a4019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:37.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a45089640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a40101280 0x7f4a401a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a4588a640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a401008d0 0x7f4a401a03d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b4a0 tx=0x7f4a3800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a3effd640 1 -- 172.21.15.45:0/3296118990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38014070 con 0x7f4a401008d0 2024-09-10T10:02:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4019afb0 con 0x7f4a401008d0 2024-09-10T10:02:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a3effd640 1 -- 172.21.15.45:0/3296118990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a38010af0 con 0x7f4a401008d0 2024-09-10T10:02:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a3effd640 1 -- 172.21.15.45:0/3296118990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38017d10 con 0x7f4a401008d0 2024-09-10T10:02:37.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.467+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a401a72a0 con 0x7f4a401008d0 2024-09-10T10:02:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.468+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a40100cb0 con 0x7f4a401008d0 2024-09-10T10:02:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.469+0000 7f4a3effd640 1 -- 172.21.15.45:0/3296118990 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4a38017070 con 0x7f4a401008d0 2024-09-10T10:02:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.470+0000 7f4a3effd640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4a280a2c90 0x7f4a280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.470+0000 7f4a45089640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4a280a2c90 0x7f4a280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.470+0000 7f4a3effd640 1 -- 172.21.15.45:0/3296118990 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4a380ce520 con 0x7f4a401008d0 2024-09-10T10:02:37.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.471+0000 7f4a45089640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4a280a2c90 0x7f4a280a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a34000c00 tx=0x7f4a34032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:37.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.474+0000 7f4a3effd640 1 -- 172.21.15.45:0/3296118990 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a3809b3e0 con 0x7f4a401008d0 2024-09-10T10:02:37.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.648+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a40062630 con 0x7f4a280a2c90 2024-09-10T10:02:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:37 smithi045 ceph-mon[31114]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:37.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:37 smithi045 ceph-mon[31114]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:37.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.657+0000 7f4a3effd640 1 -- 172.21.15.45:0/3296118990 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258638 (secure 0 0 0) 0x7f4a40062630 con 0x7f4a280a2c90 2024-09-10T10:02:37.661 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-09-10T10:02:37.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4a280a2c90 msgr2=0x7f4a280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4a280a2c90 0x7f4a280a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a34000c00 tx=0x7f4a34032d50 comp rx=0 tx=0).stop 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a401008d0 msgr2=0x7f4a401a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a401008d0 0x7f4a401a03d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b4a0 tx=0x7f4a3800b960 comp rx=0 tx=0).stop 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 shutdown_connections 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f4a280a2c90 0x7f4a280a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f4a4010e480 0x7f4a4019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a40101280 0x7f4a401a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 --2- 172.21.15.45:0/3296118990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a401008d0 0x7f4a401a03d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 >> 172.21.15.45:0/3296118990 conn(0x7f4a400fc7a0 msgr2=0x7f4a4010f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 shutdown_connections 2024-09-10T10:02:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:37.661+0000 7f4a47aee640 1 -- 172.21.15.45:0/3296118990 wait complete. 2024-09-10T10:02:37.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:37 smithi162 ceph-mon[34963]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:37 smithi162 ceph-mon[34963]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:37 smithi047 ceph-mon[35244]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:37 smithi047 ceph-mon[35244]: from='mgr.14150 172.21.15.45:0/4041615470' entity='mgr.x' 2024-09-10T10:02:38.327 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-10T10:02:36.629174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654219,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":251,"num_read_kb":240,"num_write":266,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357620,"kb_used_data":12504,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584204,"statfs":{"total":1151940427776,"available":1151574224896,"internally_reserved":0,"allocated":12804096,"data_stored":7215969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003141"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335498+0000","last_change":"2024-09-10T10:01:34.253963+0000","last_active":"2024-09-10T10:01:50.335498+0000","last_peered":"2024-09-10T10:01:50.335498+0000","last_clean":"2024-09-10T10:01:50.335498+0000","last_became_active":"2024-09-10T10:01:34.252547+0000","last_became_peered":"2024-09-10T10:01:34.252547+0000","last_unstale":"2024-09-10T10:01:50.335498+0000","last_undegraded":"2024-09-10T10:01:50.335498+0000","last_fullsized":"2024-09-10T10:01:50.335498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:55.761035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321219+0000","last_change":"2024-09-10T10:01:34.249329+0000","last_active":"2024-09-10T10:01:50.321219+0000","last_peered":"2024-09-10T10:01:50.321219+0000","last_clean":"2024-09-10T10:01:50.321219+0000","last_became_active":"2024-09-10T10:01:34.249036+0000","last_became_peered":"2024-09-10T10:01:34.249036+0000","last_unstale":"2024-09-10T10:01:50.321219+0000","last_undegraded":"2024-09-10T10:01:50.321219+0000","last_fullsized":"2024-09-10T10:01:50.321219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:26.244307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320430+0000","last_change":"2024-09-10T10:01:34.242770+0000","last_active":"2024-09-10T10:01:50.320430+0000","last_peered":"2024-09-10T10:01:50.320430+0000","last_clean":"2024-09-10T10:01:50.320430+0000","last_became_active":"2024-09-10T10:01:34.242465+0000","last_became_peered":"2024-09-10T10:01:34.242465+0000","last_unstale":"2024-09-10T10:01:50.320430+0000","last_undegraded":"2024-09-10T10:01:50.320430+0000","last_fullsized":"2024-09-10T10:01:50.320430+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:32.776914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838909+0000","last_change":"2024-09-10T10:01:34.249110+0000","last_active":"2024-09-10T10:01:50.838909+0000","last_peered":"2024-09-10T10:01:50.838909+0000","last_clean":"2024-09-10T10:01:50.838909+0000","last_became_active":"2024-09-10T10:01:34.248758+0000","last_became_peered":"2024-09-10T10:01:34.248758+0000","last_unstale":"2024-09-10T10:01:50.838909+0000","last_undegraded":"2024-09-10T10:01:50.838909+0000","last_fullsized":"2024-09-10T10:01:50.838909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:55:51.622686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338095+0000","last_change":"2024-09-10T10:01:34.248799+0000","last_active":"2024-09-10T10:01:50.338095+0000","last_peered":"2024-09-10T10:01:50.338095+0000","last_clean":"2024-09-10T10:01:50.338095+0000","last_became_active":"2024-09-10T10:01:34.246797+0000","last_became_peered":"2024-09-10T10:01:34.246797+0000","last_unstale":"2024-09-10T10:01:50.338095+0000","last_undegraded":"2024-09-10T10:01:50.338095+0000","last_fullsized":"2024-09-10T10:01:50.338095+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:33.887465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320541+0000","last_change":"2024-09-10T10:01:34.252139+0000","last_active":"2024-09-10T10:01:50.320541+0000","last_peered":"2024-09-10T10:01:50.320541+0000","last_clean":"2024-09-10T10:01:50.320541+0000","last_became_active":"2024-09-10T10:01:34.251823+0000","last_became_peered":"2024-09-10T10:01:34.251823+0000","last_unstale":"2024-09-10T10:01:50.320541+0000","last_undegraded":"2024-09-10T10:01:50.320541+0000","last_fullsized":"2024-09-10T10:01:50.320541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:31.642428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337590+0000","last_change":"2024-09-10T10:01:34.258097+0000","last_active":"2024-09-10T10:01:50.337590+0000","last_peered":"2024-09-10T10:01:50.337590+0000","last_clean":"2024-09-10T10:01:50.337590+0000","last_became_active":"2024-09-10T10:01:34.257717+0000","last_became_peered":"2024-09-10T10:01:34.257717+0000","last_unstale":"2024-09-10T10:01:50.337590+0000","last_undegraded":"2024-09-10T10:01:50.337590+0000","last_fullsized":"2024-09-10T10:01:50.337590+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:16.280503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319894+0000","last_change":"2024-09-10T10:01:34.249901+0000","last_active":"2024-09-10T10:01:50.319894+0000","last_peered":"2024-09-10T10:01:50.319894+0000","last_clean":"2024-09-10T10:01:50.319894+0000","last_became_active":"2024-09-10T10:01:34.249361+0000","last_became_peered":"2024-09-10T10:01:34.249361+0000","last_unstale":"2024-09-10T10:01:50.319894+0000","last_undegraded":"2024-09-10T10:01:50.319894+0000","last_fullsized":"2024-09-10T10:01:50.319894+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:42.282621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.711941+0000","last_change":"2024-09-10T10:01:50.711491+0000","last_active":"2024-09-10T10:01:50.711941+0000","last_peered":"2024-09-10T10:01:50.711941+0000","last_clean":"2024-09-10T10:01:50.711941+0000","last_became_active":"2024-09-10T10:01:50.711162+0000","last_became_peered":"2024-09-10T10:01:50.711162+0000","last_unstale":"2024-09-10T10:01:50.711941+0000","last_undegraded":"2024-09-10T10:01:50.711941+0000","last_fullsized":"2024-09-10T10:01:50.711941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:08.502816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320934+0000","last_change":"2024-09-10T10:01:34.252538+0000","last_active":"2024-09-10T10:01:50.320934+0000","last_peered":"2024-09-10T10:01:50.320934+0000","last_clean":"2024-09-10T10:01:50.320934+0000","last_became_active":"2024-09-10T10:01:34.251384+0000","last_became_peered":"2024-09-10T10:01:34.251384+0000","last_unstale":"2024-09-10T10:01:50.320934+0000","last_undegraded":"2024-09-10T10:01:50.320934+0000","last_fullsized":"2024-09-10T10:01:50.320934+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:39.990996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338061+0000","last_change":"2024-09-10T10:01:34.254262+0000","last_active":"2024-09-10T10:01:50.338061+0000","last_peered":"2024-09-10T10:01:50.338061+0000","last_clean":"2024-09-10T10:01:50.338061+0000","last_became_active":"2024-09-10T10:01:34.253907+0000","last_became_peered":"2024-09-10T10:01:34.253907+0000","last_unstale":"2024-09-10T10:01:50.338061+0000","last_undegraded":"2024-09-10T10:01:50.338061+0000","last_fullsized":"2024-09-10T10:01:50.338061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:38.681367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838921+0000","last_change":"2024-09-10T10:01:34.252194+0000","last_active":"2024-09-10T10:01:50.838921+0000","last_peered":"2024-09-10T10:01:50.838921+0000","last_clean":"2024-09-10T10:01:50.838921+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.838921+0000","last_undegraded":"2024-09-10T10:01:50.838921+0000","last_fullsized":"2024-09-10T10:01:50.838921+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:47.399759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332280+0000","last_change":"2024-09-10T10:01:34.251795+0000","last_active":"2024-09-10T10:01:50.332280+0000","last_peered":"2024-09-10T10:01:50.332280+0000","last_clean":"2024-09-10T10:01:50.332280+0000","last_became_active":"2024-09-10T10:01:34.251403+0000","last_became_peered":"2024-09-10T10:01:34.251403+0000","last_unstale":"2024-09-10T10:01:50.332280+0000","last_undegraded":"2024-09-10T10:01:50.332280+0000","last_fullsized":"2024-09-10T10:01:50.332280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:11.650068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321079+0000","last_change":"2024-09-10T10:01:34.252718+0000","last_active":"2024-09-10T10:01:50.321079+0000","last_peered":"2024-09-10T10:01:50.321079+0000","last_clean":"2024-09-10T10:01:50.321079+0000","last_became_active":"2024-09-10T10:01:34.252415+0000","last_became_peered":"2024-09-10T10:01:34.252415+0000","last_unstale":"2024-09-10T10:01:50.321079+0000","last_undegraded":"2024-09-10T10:01:50.321079+0000","last_fullsized":"2024-09-10T10:01:50.321079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:51.057191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320828+0000","last_change":"2024-09-10T10:01:34.252237+0000","last_active":"2024-09-10T10:01:50.320828+0000","last_peered":"2024-09-10T10:01:50.320828+0000","last_clean":"2024-09-10T10:01:50.320828+0000","last_became_active":"2024-09-10T10:01:34.251909+0000","last_became_peered":"2024-09-10T10:01:34.251909+0000","last_unstale":"2024-09-10T10:01:50.320828+0000","last_undegraded":"2024-09-10T10:01:50.320828+0000","last_fullsized":"2024-09-10T10:01:50.320828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:15.150503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318205+0000","last_change":"2024-09-10T10:01:34.259107+0000","last_active":"2024-09-10T10:01:50.318205+0000","last_peered":"2024-09-10T10:01:50.318205+0000","last_clean":"2024-09-10T10:01:50.318205+0000","last_became_active":"2024-09-10T10:01:34.258230+0000","last_became_peered":"2024-09-10T10:01:34.258230+0000","last_unstale":"2024-09-10T10:01:50.318205+0000","last_undegraded":"2024-09-10T10:01:50.318205+0000","last_fullsized":"2024-09-10T10:01:50.318205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:36.091358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335573+0000","last_change":"2024-09-10T10:01:34.254341+0000","last_active":"2024-09-10T10:01:50.335573+0000","last_peered":"2024-09-10T10:01:50.335573+0000","last_clean":"2024-09-10T10:01:50.335573+0000","last_became_active":"2024-09-10T10:01:34.251715+0000","last_became_peered":"2024-09-10T10:01:34.251715+0000","last_unstale":"2024-09-10T10:01:50.335573+0000","last_undegraded":"2024-09-10T10:01:50.335573+0000","last_fullsized":"2024-09-10T10:01:50.335573+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:43:46.668715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317729+0000","last_change":"2024-09-10T10:01:34.247950+0000","last_active":"2024-09-10T10:01:50.317729+0000","last_peered":"2024-09-10T10:01:50.317729+0000","last_clean":"2024-09-10T10:01:50.317729+0000","last_became_active":"2024-09-10T10:01:34.247659+0000","last_became_peered":"2024-09-10T10:01:34.247659+0000","last_unstale":"2024-09-10T10:01:50.317729+0000","last_undegraded":"2024-09-10T10:01:50.317729+0000","last_fullsized":"2024-09-10T10:01:50.317729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:34.113605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333284+0000","last_change":"2024-09-10T10:01:34.248153+0000","last_active":"2024-09-10T10:01:50.333284+0000","last_peered":"2024-09-10T10:01:50.333284+0000","last_clean":"2024-09-10T10:01:50.333284+0000","last_became_active":"2024-09-10T10:01:34.247773+0000","last_became_peered":"2024-09-10T10:01:34.247773+0000","last_unstale":"2024-09-10T10:01:50.333284+0000","last_undegraded":"2024-09-10T10:01:50.333284+0000","last_fullsized":"2024-09-10T10:01:50.333284+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:26:54.223519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335273+0000","last_change":"2024-09-10T10:01:34.252781+0000","last_active":"2024-09-10T10:01:50.335273+0000","last_peered":"2024-09-10T10:01:50.335273+0000","last_clean":"2024-09-10T10:01:50.335273+0000","last_became_active":"2024-09-10T10:01:34.252459+0000","last_became_peered":"2024-09-10T10:01:34.252459+0000","last_unstale":"2024-09-10T10:01:50.335273+0000","last_undegraded":"2024-09-10T10:01:50.335273+0000","last_fullsized":"2024-09-10T10:01:50.335273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:32.039769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332286+0000","last_change":"2024-09-10T10:01:34.251147+0000","last_active":"2024-09-10T10:01:50.332286+0000","last_peered":"2024-09-10T10:01:50.332286+0000","last_clean":"2024-09-10T10:01:50.332286+0000","last_became_active":"2024-09-10T10:01:34.250835+0000","last_became_peered":"2024-09-10T10:01:34.250835+0000","last_unstale":"2024-09-10T10:01:50.332286+0000","last_undegraded":"2024-09-10T10:01:50.332286+0000","last_fullsized":"2024-09-10T10:01:50.332286+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:56.914798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318417+0000","last_change":"2024-09-10T10:01:34.259300+0000","last_active":"2024-09-10T10:01:50.318417+0000","last_peered":"2024-09-10T10:01:50.318417+0000","last_clean":"2024-09-10T10:01:50.318417+0000","last_became_active":"2024-09-10T10:01:34.258192+0000","last_became_peered":"2024-09-10T10:01:34.258192+0000","last_unstale":"2024-09-10T10:01:50.318417+0000","last_undegraded":"2024-09-10T10:01:50.318417+0000","last_fullsized":"2024-09-10T10:01:50.318417+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:19.755696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319962+0000","last_change":"2024-09-10T10:01:34.244797+0000","last_active":"2024-09-10T10:01:50.319962+0000","last_peered":"2024-09-10T10:01:50.319962+0000","last_clean":"2024-09-10T10:01:50.319962+0000","last_became_active":"2024-09-10T10:01:34.244464+0000","last_became_peered":"2024-09-10T10:01:34.244464+0000","last_unstale":"2024-09-10T10:01:50.319962+0000","last_undegraded":"2024-09-10T10:01:50.319962+0000","last_fullsized":"2024-09-10T10:01:50.319962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:23.750063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710213+0000","last_change":"2024-09-10T10:01:34.248445+0000","last_active":"2024-09-10T10:01:50.710213+0000","last_peered":"2024-09-10T10:01:50.710213+0000","last_clean":"2024-09-10T10:01:50.710213+0000","last_became_active":"2024-09-10T10:01:34.248020+0000","last_became_peered":"2024-09-10T10:01:34.248020+0000","last_unstale":"2024-09-10T10:01:50.710213+0000","last_undegraded":"2024-09-10T10:01:50.710213+0000","last_fullsized":"2024-09-10T10:01:50.710213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:32.096061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339780+0000","last_change":"2024-09-10T10:01:50.339780+0000","last_active":"2024-09-10T10:01:50.339780+0000","last_peered":"2024-09-10T10:01:50.339780+0000","last_clean":"2024-09-10T10:01:50.339780+0000","last_became_active":"2024-09-10T10:01:50.339519+0000","last_became_peered":"2024-09-10T10:01:50.339519+0000","last_unstale":"2024-09-10T10:01:50.339780+0000","last_undegraded":"2024-09-10T10:01:50.339780+0000","last_fullsized":"2024-09-10T10:01:50.339780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:27.368296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333351+0000","last_change":"2024-09-10T10:01:34.244512+0000","last_active":"2024-09-10T10:01:50.333351+0000","last_peered":"2024-09-10T10:01:50.333351+0000","last_clean":"2024-09-10T10:01:50.333351+0000","last_became_active":"2024-09-10T10:01:34.244191+0000","last_became_peered":"2024-09-10T10:01:34.244191+0000","last_unstale":"2024-09-10T10:01:50.333351+0000","last_undegraded":"2024-09-10T10:01:50.333351+0000","last_fullsized":"2024-09-10T10:01:50.333351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:19.019159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335673+0000","last_change":"2024-09-10T10:01:34.248683+0000","last_active":"2024-09-10T10:01:50.335673+0000","last_peered":"2024-09-10T10:01:50.335673+0000","last_clean":"2024-09-10T10:01:50.335673+0000","last_became_active":"2024-09-10T10:01:34.248228+0000","last_became_peered":"2024-09-10T10:01:34.248228+0000","last_unstale":"2024-09-10T10:01:50.335673+0000","last_undegraded":"2024-09-10T10:01:50.335673+0000","last_fullsized":"2024-09-10T10:01:50.335673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:00.485621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332452+0000","last_change":"2024-09-10T10:01:34.254683+0000","last_active":"2024-09-10T10:01:50.332452+0000","last_peered":"2024-09-10T10:01:50.332452+0000","last_clean":"2024-09-10T10:01:50.332452+0000","last_became_active":"2024-09-10T10:01:34.254384+0000","last_became_peered":"2024-09-10T10:01:34.254384+0000","last_unstale":"2024-09-10T10:01:50.332452+0000","last_undegraded":"2024-09-10T10:01:50.332452+0000","last_fullsized":"2024-09-10T10:01:50.332452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:35.404584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321448+0000","last_change":"2024-09-10T10:01:34.254079+0000","last_active":"2024-09-10T10:01:50.321448+0000","last_peered":"2024-09-10T10:01:50.321448+0000","last_clean":"2024-09-10T10:01:50.321448+0000","last_became_active":"2024-09-10T10:01:34.253527+0000","last_became_peered":"2024-09-10T10:01:34.253527+0000","last_unstale":"2024-09-10T10:01:50.321448+0000","last_undegraded":"2024-09-10T10:01:50.321448+0000","last_fullsized":"2024-09-10T10:01:50.321448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:31.406025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332948+0000","last_change":"2024-09-10T10:01:34.244840+0000","last_active":"2024-09-10T10:01:50.332948+0000","last_peered":"2024-09-10T10:01:50.332948+0000","last_clean":"2024-09-10T10:01:50.332948+0000","last_became_active":"2024-09-10T10:01:34.244482+0000","last_became_peered":"2024-09-10T10:01:34.244482+0000","last_unstale":"2024-09-10T10:01:50.332948+0000","last_undegraded":"2024-09-10T10:01:50.332948+0000","last_fullsized":"2024-09-10T10:01:50.332948+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:50.666840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332939+0000","last_change":"2024-09-10T10:01:34.250789+0000","last_active":"2024-09-10T10:01:50.332939+0000","last_peered":"2024-09-10T10:01:50.332939+0000","last_clean":"2024-09-10T10:01:50.332939+0000","last_became_active":"2024-09-10T10:01:34.250445+0000","last_became_peered":"2024-09-10T10:01:34.250445+0000","last_unstale":"2024-09-10T10:01:50.332939+0000","last_undegraded":"2024-09-10T10:01:50.332939+0000","last_fullsized":"2024-09-10T10:01:50.332939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:00.389643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335618+0000","last_change":"2024-09-10T10:01:34.253139+0000","last_active":"2024-09-10T10:01:50.335618+0000","last_peered":"2024-09-10T10:01:50.335618+0000","last_clean":"2024-09-10T10:01:50.335618+0000","last_became_active":"2024-09-10T10:01:34.251801+0000","last_became_peered":"2024-09-10T10:01:34.251801+0000","last_unstale":"2024-09-10T10:01:50.335618+0000","last_undegraded":"2024-09-10T10:01:50.335618+0000","last_fullsized":"2024-09-10T10:01:50.335618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:33.492291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338268+0000","last_change":"2024-09-10T10:01:34.243669+0000","last_active":"2024-09-10T10:01:50.338268+0000","last_peered":"2024-09-10T10:01:50.338268+0000","last_clean":"2024-09-10T10:01:50.338268+0000","last_became_active":"2024-09-10T10:01:34.243291+0000","last_became_peered":"2024-09-10T10:01:34.243291+0000","last_unstale":"2024-09-10T10:01:50.338268+0000","last_undegraded":"2024-09-10T10:01:50.338268+0000","last_fullsized":"2024-09-10T10:01:50.338268+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:47.157495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710729+0000","last_change":"2024-09-10T10:01:32.206317+0000","last_active":"2024-09-10T10:01:50.710729+0000","last_peered":"2024-09-10T10:01:50.710729+0000","last_clean":"2024-09-10T10:01:50.710729+0000","last_became_active":"2024-09-10T10:01:32.205412+0000","last_became_peered":"2024-09-10T10:01:32.205412+0000","last_unstale":"2024-09-10T10:01:50.710729+0000","last_undegraded":"2024-09-10T10:01:50.710729+0000","last_fullsized":"2024-09-10T10:01:50.710729+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:07.489923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710763+0000","last_change":"2024-09-10T10:01:34.254783+0000","last_active":"2024-09-10T10:01:50.710763+0000","last_peered":"2024-09-10T10:01:50.710763+0000","last_clean":"2024-09-10T10:01:50.710763+0000","last_became_active":"2024-09-10T10:01:34.254465+0000","last_became_peered":"2024-09-10T10:01:34.254465+0000","last_unstale":"2024-09-10T10:01:50.710763+0000","last_undegraded":"2024-09-10T10:01:50.710763+0000","last_fullsized":"2024-09-10T10:01:50.710763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:57.983265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320605+0000","last_change":"2024-09-10T10:01:32.204551+0000","last_active":"2024-09-10T10:01:50.320605+0000","last_peered":"2024-09-10T10:01:50.320605+0000","last_clean":"2024-09-10T10:01:50.320605+0000","last_became_active":"2024-09-10T10:01:32.204214+0000","last_became_peered":"2024-09-10T10:01:32.204214+0000","last_unstale":"2024-09-10T10:01:50.320605+0000","last_undegraded":"2024-09-10T10:01:50.320605+0000","last_fullsized":"2024-09-10T10:01:50.320605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:50.765757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332156+0000","last_change":"2024-09-10T10:01:32.205907+0000","last_active":"2024-09-10T10:01:50.332156+0000","last_peered":"2024-09-10T10:01:50.332156+0000","last_clean":"2024-09-10T10:01:50.332156+0000","last_became_active":"2024-09-10T10:01:32.205523+0000","last_became_peered":"2024-09-10T10:01:32.205523+0000","last_unstale":"2024-09-10T10:01:50.332156+0000","last_undegraded":"2024-09-10T10:01:50.332156+0000","last_fullsized":"2024-09-10T10:01:50.332156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:14.540234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.319983+0000","last_change":"2024-09-10T10:01:34.249826+0000","last_active":"2024-09-10T10:01:50.319983+0000","last_peered":"2024-09-10T10:01:50.319983+0000","last_clean":"2024-09-10T10:01:50.319983+0000","last_became_active":"2024-09-10T10:01:34.248940+0000","last_became_peered":"2024-09-10T10:01:34.248940+0000","last_unstale":"2024-09-10T10:01:50.319983+0000","last_undegraded":"2024-09-10T10:01:50.319983+0000","last_fullsized":"2024-09-10T10:01:50.319983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:49.125909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710836+0000","last_change":"2024-09-10T10:01:32.205559+0000","last_active":"2024-09-10T10:01:50.710836+0000","last_peered":"2024-09-10T10:01:50.710836+0000","last_clean":"2024-09-10T10:01:50.710836+0000","last_became_active":"2024-09-10T10:01:32.205251+0000","last_became_peered":"2024-09-10T10:01:32.205251+0000","last_unstale":"2024-09-10T10:01:50.710836+0000","last_undegraded":"2024-09-10T10:01:50.710836+0000","last_fullsized":"2024-09-10T10:01:50.710836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:11.442234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317525+0000","last_change":"2024-09-10T10:01:34.245852+0000","last_active":"2024-09-10T10:01:50.317525+0000","last_peered":"2024-09-10T10:01:50.317525+0000","last_clean":"2024-09-10T10:01:50.317525+0000","last_became_active":"2024-09-10T10:01:34.245487+0000","last_became_peered":"2024-09-10T10:01:34.245487+0000","last_unstale":"2024-09-10T10:01:50.317525+0000","last_undegraded":"2024-09-10T10:01:50.317525+0000","last_fullsized":"2024-09-10T10:01:50.317525+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:01.995443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320839+0000","last_change":"2024-09-10T10:01:32.202754+0000","last_active":"2024-09-10T10:01:50.320839+0000","last_peered":"2024-09-10T10:01:50.320839+0000","last_clean":"2024-09-10T10:01:50.320839+0000","last_became_active":"2024-09-10T10:01:32.202360+0000","last_became_peered":"2024-09-10T10:01:32.202360+0000","last_unstale":"2024-09-10T10:01:50.320839+0000","last_undegraded":"2024-09-10T10:01:50.320839+0000","last_fullsized":"2024-09-10T10:01:50.320839+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:04.864276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318627+0000","last_change":"2024-09-10T10:01:34.245670+0000","last_active":"2024-09-10T10:01:50.318627+0000","last_peered":"2024-09-10T10:01:50.318627+0000","last_clean":"2024-09-10T10:01:50.318627+0000","last_became_active":"2024-09-10T10:01:34.244922+0000","last_became_peered":"2024-09-10T10:01:34.244922+0000","last_unstale":"2024-09-10T10:01:50.318627+0000","last_undegraded":"2024-09-10T10:01:50.318627+0000","last_fullsized":"2024-09-10T10:01:50.318627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:39:05.442144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338108+0000","last_change":"2024-09-10T10:01:32.204658+0000","last_active":"2024-09-10T10:01:50.338108+0000","last_peered":"2024-09-10T10:01:50.338108+0000","last_clean":"2024-09-10T10:01:50.338108+0000","last_became_active":"2024-09-10T10:01:32.204310+0000","last_became_peered":"2024-09-10T10:01:32.204310+0000","last_unstale":"2024-09-10T10:01:50.338108+0000","last_undegraded":"2024-09-10T10:01:50.338108+0000","last_fullsized":"2024-09-10T10:01:50.338108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:16.446977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.712239+0000","last_change":"2024-09-10T10:01:50.712239+0000","last_active":"2024-09-10T10:01:50.712239+0000","last_peered":"2024-09-10T10:01:50.712239+0000","last_clean":"2024-09-10T10:01:50.712239+0000","last_became_active":"2024-09-10T10:01:50.711826+0000","last_became_peered":"2024-09-10T10:01:50.711826+0000","last_unstale":"2024-09-10T10:01:50.712239+0000","last_undegraded":"2024-09-10T10:01:50.712239+0000","last_fullsized":"2024-09-10T10:01:50.712239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:54.670359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710657+0000","last_change":"2024-09-10T10:01:32.530649+0000","last_active":"2024-09-10T10:01:50.710657+0000","last_peered":"2024-09-10T10:01:50.710657+0000","last_clean":"2024-09-10T10:01:50.710657+0000","last_became_active":"2024-09-10T10:01:32.530157+0000","last_became_peered":"2024-09-10T10:01:32.530157+0000","last_unstale":"2024-09-10T10:01:50.710657+0000","last_undegraded":"2024-09-10T10:01:50.710657+0000","last_fullsized":"2024-09-10T10:01:50.710657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:30.235163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339461+0000","last_change":"2024-09-10T10:01:50.339461+0000","last_active":"2024-09-10T10:01:50.339461+0000","last_peered":"2024-09-10T10:01:50.339461+0000","last_clean":"2024-09-10T10:01:50.339461+0000","last_became_active":"2024-09-10T10:01:50.339158+0000","last_became_peered":"2024-09-10T10:01:50.339158+0000","last_unstale":"2024-09-10T10:01:50.339461+0000","last_undegraded":"2024-09-10T10:01:50.339461+0000","last_fullsized":"2024-09-10T10:01:50.339461+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:11.733302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'27","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:02:31.357860+0000","last_change":"2024-09-10T10:01:32.205933+0000","last_active":"2024-09-10T10:02:31.357860+0000","last_peered":"2024-09-10T10:02:31.357860+0000","last_clean":"2024-09-10T10:02:31.357860+0000","last_became_active":"2024-09-10T10:01:32.205592+0000","last_became_peered":"2024-09-10T10:01:32.205592+0000","last_unstale":"2024-09-10T10:02:31.357860+0000","last_undegraded":"2024-09-10T10:02:31.357860+0000","last_fullsized":"2024-09-10T10:02:31.357860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:29.107388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62819,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":109,"num_read_kb":115,"num_write":27,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338800+0000","last_change":"2024-09-10T10:01:34.249720+0000","last_active":"2024-09-10T10:01:50.338800+0000","last_peered":"2024-09-10T10:01:50.338800+0000","last_clean":"2024-09-10T10:01:50.338800+0000","last_became_active":"2024-09-10T10:01:34.249285+0000","last_became_peered":"2024-09-10T10:01:34.249285+0000","last_unstale":"2024-09-10T10:01:50.338800+0000","last_undegraded":"2024-09-10T10:01:50.338800+0000","last_fullsized":"2024-09-10T10:01:50.338800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:30.787344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333124+0000","last_change":"2024-09-10T10:01:32.202622+0000","last_active":"2024-09-10T10:01:50.333124+0000","last_peered":"2024-09-10T10:01:50.333124+0000","last_clean":"2024-09-10T10:01:50.333124+0000","last_became_active":"2024-09-10T10:01:32.200870+0000","last_became_peered":"2024-09-10T10:01:32.200870+0000","last_unstale":"2024-09-10T10:01:50.333124+0000","last_undegraded":"2024-09-10T10:01:50.333124+0000","last_fullsized":"2024-09-10T10:01:50.333124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:48.562971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332189+0000","last_change":"2024-09-10T10:01:34.252622+0000","last_active":"2024-09-10T10:01:50.332189+0000","last_peered":"2024-09-10T10:01:50.332189+0000","last_clean":"2024-09-10T10:01:50.332189+0000","last_became_active":"2024-09-10T10:01:34.252271+0000","last_became_peered":"2024-09-10T10:01:34.252271+0000","last_unstale":"2024-09-10T10:01:50.332189+0000","last_undegraded":"2024-09-10T10:01:50.332189+0000","last_fullsized":"2024-09-10T10:01:50.332189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:03.269962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710592+0000","last_change":"2024-09-10T10:01:32.206133+0000","last_active":"2024-09-10T10:01:50.710592+0000","last_peered":"2024-09-10T10:01:50.710592+0000","last_clean":"2024-09-10T10:01:50.710592+0000","last_became_active":"2024-09-10T10:01:32.205747+0000","last_became_peered":"2024-09-10T10:01:32.205747+0000","last_unstale":"2024-09-10T10:01:50.710592+0000","last_undegraded":"2024-09-10T10:01:50.710592+0000","last_fullsized":"2024-09-10T10:01:50.710592+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:30.515614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337830+0000","last_change":"2024-09-10T10:01:34.257702+0000","last_active":"2024-09-10T10:01:50.337830+0000","last_peered":"2024-09-10T10:01:50.337830+0000","last_clean":"2024-09-10T10:01:50.337830+0000","last_became_active":"2024-09-10T10:01:34.257366+0000","last_became_peered":"2024-09-10T10:01:34.257366+0000","last_unstale":"2024-09-10T10:01:50.337830+0000","last_undegraded":"2024-09-10T10:01:50.337830+0000","last_fullsized":"2024-09-10T10:01:50.337830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:51.772522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838725+0000","last_change":"2024-09-10T10:01:32.205527+0000","last_active":"2024-09-10T10:01:50.838725+0000","last_peered":"2024-09-10T10:01:50.838725+0000","last_clean":"2024-09-10T10:01:50.838725+0000","last_became_active":"2024-09-10T10:01:32.205162+0000","last_became_peered":"2024-09-10T10:01:32.205162+0000","last_unstale":"2024-09-10T10:01:50.838725+0000","last_undegraded":"2024-09-10T10:01:50.838725+0000","last_fullsized":"2024-09-10T10:01:50.838725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:28.149331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337435+0000","last_change":"2024-09-10T10:01:34.250691+0000","last_active":"2024-09-10T10:01:50.337435+0000","last_peered":"2024-09-10T10:01:50.337435+0000","last_clean":"2024-09-10T10:01:50.337435+0000","last_became_active":"2024-09-10T10:01:34.249303+0000","last_became_peered":"2024-09-10T10:01:34.249303+0000","last_unstale":"2024-09-10T10:01:50.337435+0000","last_undegraded":"2024-09-10T10:01:50.337435+0000","last_fullsized":"2024-09-10T10:01:50.337435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:47.413567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'8","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338071+0000","last_change":"2024-09-10T10:01:32.530043+0000","last_active":"2024-09-10T10:01:50.338071+0000","last_peered":"2024-09-10T10:01:50.338071+0000","last_clean":"2024-09-10T10:01:50.338071+0000","last_became_active":"2024-09-10T10:01:32.529502+0000","last_became_peered":"2024-09-10T10:01:32.529502+0000","last_unstale":"2024-09-10T10:01:50.338071+0000","last_undegraded":"2024-09-10T10:01:50.338071+0000","last_fullsized":"2024-09-10T10:01:50.338071+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:26.133771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710304+0000","last_change":"2024-09-10T10:01:34.254722+0000","last_active":"2024-09-10T10:01:50.710304+0000","last_peered":"2024-09-10T10:01:50.710304+0000","last_clean":"2024-09-10T10:01:50.710304+0000","last_became_active":"2024-09-10T10:01:34.254390+0000","last_became_peered":"2024-09-10T10:01:34.254390+0000","last_unstale":"2024-09-10T10:01:50.710304+0000","last_undegraded":"2024-09-10T10:01:50.710304+0000","last_fullsized":"2024-09-10T10:01:50.710304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:37.261256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.339061+0000","last_change":"2024-09-10T10:01:32.200744+0000","last_active":"2024-09-10T10:01:50.339061+0000","last_peered":"2024-09-10T10:01:50.339061+0000","last_clean":"2024-09-10T10:01:50.339061+0000","last_became_active":"2024-09-10T10:01:32.199906+0000","last_became_peered":"2024-09-10T10:01:32.199906+0000","last_unstale":"2024-09-10T10:01:50.339061+0000","last_undegraded":"2024-09-10T10:01:50.339061+0000","last_fullsized":"2024-09-10T10:01:50.339061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:17.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.840135+0000","last_change":"2024-09-10T10:01:50.840135+0000","last_active":"2024-09-10T10:01:50.840135+0000","last_peered":"2024-09-10T10:01:50.840135+0000","last_clean":"2024-09-10T10:01:50.840135+0000","last_became_active":"2024-09-10T10:01:50.839693+0000","last_became_peered":"2024-09-10T10:01:50.839693+0000","last_unstale":"2024-09-10T10:01:50.840135+0000","last_undegraded":"2024-09-10T10:01:50.840135+0000","last_fullsized":"2024-09-10T10:01:50.840135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:01.942607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332542+0000","last_change":"2024-09-10T10:01:32.200383+0000","last_active":"2024-09-10T10:01:50.332542+0000","last_peered":"2024-09-10T10:01:50.332542+0000","last_clean":"2024-09-10T10:01:50.332542+0000","last_became_active":"2024-09-10T10:01:32.199926+0000","last_became_peered":"2024-09-10T10:01:32.199926+0000","last_unstale":"2024-09-10T10:01:50.332542+0000","last_undegraded":"2024-09-10T10:01:50.332542+0000","last_fullsized":"2024-09-10T10:01:50.332542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:29.645863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338221+0000","last_change":"2024-09-10T10:01:34.250077+0000","last_active":"2024-09-10T10:01:50.338221+0000","last_peered":"2024-09-10T10:01:50.338221+0000","last_clean":"2024-09-10T10:01:50.338221+0000","last_became_active":"2024-09-10T10:01:34.249697+0000","last_became_peered":"2024-09-10T10:01:34.249697+0000","last_unstale":"2024-09-10T10:01:50.338221+0000","last_undegraded":"2024-09-10T10:01:50.338221+0000","last_fullsized":"2024-09-10T10:01:50.338221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:25.068115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710730+0000","last_change":"2024-09-10T10:01:32.200235+0000","last_active":"2024-09-10T10:01:50.710730+0000","last_peered":"2024-09-10T10:01:50.710730+0000","last_clean":"2024-09-10T10:01:50.710730+0000","last_became_active":"2024-09-10T10:01:32.199920+0000","last_became_peered":"2024-09-10T10:01:32.199920+0000","last_unstale":"2024-09-10T10:01:50.710730+0000","last_undegraded":"2024-09-10T10:01:50.710730+0000","last_fullsized":"2024-09-10T10:01:50.710730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:10.980054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338054+0000","last_change":"2024-09-10T10:01:34.256937+0000","last_active":"2024-09-10T10:01:50.338054+0000","last_peered":"2024-09-10T10:01:50.338054+0000","last_clean":"2024-09-10T10:01:50.338054+0000","last_became_active":"2024-09-10T10:01:34.256376+0000","last_became_peered":"2024-09-10T10:01:34.256376+0000","last_unstale":"2024-09-10T10:01:50.338054+0000","last_undegraded":"2024-09-10T10:01:50.338054+0000","last_fullsized":"2024-09-10T10:01:50.338054+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:07.386683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333436+0000","last_change":"2024-09-10T10:01:32.530794+0000","last_active":"2024-09-10T10:01:50.333436+0000","last_peered":"2024-09-10T10:01:50.333436+0000","last_clean":"2024-09-10T10:01:50.333436+0000","last_became_active":"2024-09-10T10:01:32.530335+0000","last_became_peered":"2024-09-10T10:01:32.530335+0000","last_unstale":"2024-09-10T10:01:50.333436+0000","last_undegraded":"2024-09-10T10:01:50.333436+0000","last_fullsized":"2024-09-10T10:01:50.333436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:40.883029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338123+0000","last_change":"2024-09-10T10:01:34.250228+0000","last_active":"2024-09-10T10:01:50.338123+0000","last_peered":"2024-09-10T10:01:50.338123+0000","last_clean":"2024-09-10T10:01:50.338123+0000","last_became_active":"2024-09-10T10:01:34.249901+0000","last_became_peered":"2024-09-10T10:01:34.249901+0000","last_unstale":"2024-09-10T10:01:50.338123+0000","last_undegraded":"2024-09-10T10:01:50.338123+0000","last_fullsized":"2024-09-10T10:01:50.338123+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:10.710185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337584+0000","last_change":"2024-09-10T10:00:23.346149+0000","last_active":"2024-09-10T10:01:50.337584+0000","last_peered":"2024-09-10T10:01:50.337584+0000","last_clean":"2024-09-10T10:01:50.337584+0000","last_became_active":"2024-09-10T10:00:23.345608+0000","last_became_peered":"2024-09-10T10:00:23.345608+0000","last_unstale":"2024-09-10T10:01:50.337584+0000","last_undegraded":"2024-09-10T10:01:50.337584+0000","last_fullsized":"2024-09-10T10:01:50.337584+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:57:03.194756+0000","last_clean_scrub_stamp":"2024-09-10T09:57:03.194756+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:14.092838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.838755+0000","last_change":"2024-09-10T10:01:32.200726+0000","last_active":"2024-09-10T10:01:50.838755+0000","last_peered":"2024-09-10T10:01:50.838755+0000","last_clean":"2024-09-10T10:01:50.838755+0000","last_became_active":"2024-09-10T10:01:32.200163+0000","last_became_peered":"2024-09-10T10:01:32.200163+0000","last_unstale":"2024-09-10T10:01:50.838755+0000","last_undegraded":"2024-09-10T10:01:50.838755+0000","last_fullsized":"2024-09-10T10:01:50.838755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:05.300476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320505+0000","last_change":"2024-09-10T10:01:34.243258+0000","last_active":"2024-09-10T10:01:50.320505+0000","last_peered":"2024-09-10T10:01:50.320505+0000","last_clean":"2024-09-10T10:01:50.320505+0000","last_became_active":"2024-09-10T10:01:34.242060+0000","last_became_peered":"2024-09-10T10:01:34.242060+0000","last_unstale":"2024-09-10T10:01:50.320505+0000","last_undegraded":"2024-09-10T10:01:50.320505+0000","last_fullsized":"2024-09-10T10:01:50.320505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:02.910538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337582+0000","last_change":"2024-09-10T10:01:32.205174+0000","last_active":"2024-09-10T10:01:50.337582+0000","last_peered":"2024-09-10T10:01:50.337582+0000","last_clean":"2024-09-10T10:01:50.337582+0000","last_became_active":"2024-09-10T10:01:32.204121+0000","last_became_peered":"2024-09-10T10:01:32.204121+0000","last_unstale":"2024-09-10T10:01:50.337582+0000","last_undegraded":"2024-09-10T10:01:50.337582+0000","last_fullsized":"2024-09-10T10:01:50.337582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:42.533591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332204+0000","last_change":"2024-09-10T10:01:34.254895+0000","last_active":"2024-09-10T10:01:50.332204+0000","last_peered":"2024-09-10T10:01:50.332204+0000","last_clean":"2024-09-10T10:01:50.332204+0000","last_became_active":"2024-09-10T10:01:34.254572+0000","last_became_peered":"2024-09-10T10:01:34.254572+0000","last_unstale":"2024-09-10T10:01:50.332204+0000","last_undegraded":"2024-09-10T10:01:50.332204+0000","last_fullsized":"2024-09-10T10:01:50.332204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:44.194941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332304+0000","last_change":"2024-09-10T10:01:32.530093+0000","last_active":"2024-09-10T10:01:50.332304+0000","last_peered":"2024-09-10T10:01:50.332304+0000","last_clean":"2024-09-10T10:01:50.332304+0000","last_became_active":"2024-09-10T10:01:32.529756+0000","last_became_peered":"2024-09-10T10:01:32.529756+0000","last_unstale":"2024-09-10T10:01:50.332304+0000","last_undegraded":"2024-09-10T10:01:50.332304+0000","last_fullsized":"2024-09-10T10:01:50.332304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:23.520549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710348+0000","last_change":"2024-09-10T10:01:34.255210+0000","last_active":"2024-09-10T10:01:50.710348+0000","last_peered":"2024-09-10T10:01:50.710348+0000","last_clean":"2024-09-10T10:01:50.710348+0000","last_became_active":"2024-09-10T10:01:34.254861+0000","last_became_peered":"2024-09-10T10:01:34.254861+0000","last_unstale":"2024-09-10T10:01:50.710348+0000","last_undegraded":"2024-09-10T10:01:50.710348+0000","last_fullsized":"2024-09-10T10:01:50.710348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:10.332758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337852+0000","last_change":"2024-09-10T10:01:32.204622+0000","last_active":"2024-09-10T10:01:50.337852+0000","last_peered":"2024-09-10T10:01:50.337852+0000","last_clean":"2024-09-10T10:01:50.337852+0000","last_became_active":"2024-09-10T10:01:32.204236+0000","last_became_peered":"2024-09-10T10:01:32.204236+0000","last_unstale":"2024-09-10T10:01:50.337852+0000","last_undegraded":"2024-09-10T10:01:50.337852+0000","last_fullsized":"2024-09-10T10:01:50.337852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:59.402854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320679+0000","last_change":"2024-09-10T10:01:34.244790+0000","last_active":"2024-09-10T10:01:50.320679+0000","last_peered":"2024-09-10T10:01:50.320679+0000","last_clean":"2024-09-10T10:01:50.320679+0000","last_became_active":"2024-09-10T10:01:34.244443+0000","last_became_peered":"2024-09-10T10:01:34.244443+0000","last_unstale":"2024-09-10T10:01:50.320679+0000","last_undegraded":"2024-09-10T10:01:50.320679+0000","last_fullsized":"2024-09-10T10:01:50.320679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:55.913475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.529774+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.529410+0000","last_became_peered":"2024-09-10T10:01:32.529410+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:01.433681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335478+0000","last_change":"2024-09-10T10:01:50.335478+0000","last_active":"2024-09-10T10:01:50.335478+0000","last_peered":"2024-09-10T10:01:50.335478+0000","last_clean":"2024-09-10T10:01:50.335478+0000","last_became_active":"2024-09-10T10:01:50.332726+0000","last_became_peered":"2024-09-10T10:01:50.332726+0000","last_unstale":"2024-09-10T10:01:50.335478+0000","last_undegraded":"2024-09-10T10:01:50.335478+0000","last_fullsized":"2024-09-10T10:01:50.335478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:04:10.726066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338880+0000","last_change":"2024-09-10T10:01:32.530401+0000","last_active":"2024-09-10T10:01:50.338880+0000","last_peered":"2024-09-10T10:01:50.338880+0000","last_clean":"2024-09-10T10:01:50.338880+0000","last_became_active":"2024-09-10T10:01:32.529928+0000","last_became_peered":"2024-09-10T10:01:32.529928+0000","last_unstale":"2024-09-10T10:01:50.338880+0000","last_undegraded":"2024-09-10T10:01:50.338880+0000","last_fullsized":"2024-09-10T10:01:50.338880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:59:18.978923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.332474+0000","last_change":"2024-09-10T10:01:34.245777+0000","last_active":"2024-09-10T10:01:50.332474+0000","last_peered":"2024-09-10T10:01:50.332474+0000","last_clean":"2024-09-10T10:01:50.332474+0000","last_became_active":"2024-09-10T10:01:34.245431+0000","last_became_peered":"2024-09-10T10:01:34.245431+0000","last_unstale":"2024-09-10T10:01:50.332474+0000","last_undegraded":"2024-09-10T10:01:50.332474+0000","last_fullsized":"2024-09-10T10:01:50.332474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:17.754117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'9","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:56.246544+0000","last_change":"2024-09-10T10:01:32.201916+0000","last_active":"2024-09-10T10:01:56.246544+0000","last_peered":"2024-09-10T10:01:56.246544+0000","last_clean":"2024-09-10T10:01:56.246544+0000","last_became_active":"2024-09-10T10:01:32.201540+0000","last_became_peered":"2024-09-10T10:01:32.201540+0000","last_unstale":"2024-09-10T10:01:56.246544+0000","last_undegraded":"2024-09-10T10:01:56.246544+0000","last_fullsized":"2024-09-10T10:01:56.246544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:41.862673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320509+0000","last_change":"2024-09-10T10:01:34.251334+0000","last_active":"2024-09-10T10:01:50.320509+0000","last_peered":"2024-09-10T10:01:50.320509+0000","last_clean":"2024-09-10T10:01:50.320509+0000","last_became_active":"2024-09-10T10:01:34.251032+0000","last_became_peered":"2024-09-10T10:01:34.251032+0000","last_unstale":"2024-09-10T10:01:50.320509+0000","last_undegraded":"2024-09-10T10:01:50.320509+0000","last_fullsized":"2024-09-10T10:01:50.320509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:26.437259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.839313+0000","last_change":"2024-09-10T10:01:32.255074+0000","last_active":"2024-09-10T10:01:50.839313+0000","last_peered":"2024-09-10T10:01:50.839313+0000","last_clean":"2024-09-10T10:01:50.839313+0000","last_became_active":"2024-09-10T10:01:32.254637+0000","last_became_peered":"2024-09-10T10:01:32.254637+0000","last_unstale":"2024-09-10T10:01:50.839313+0000","last_undegraded":"2024-09-10T10:01:50.839313+0000","last_fullsized":"2024-09-10T10:01:50.839313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:35.392816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.321297+0000","last_change":"2024-09-10T10:01:34.253770+0000","last_active":"2024-09-10T10:01:50.321297+0000","last_peered":"2024-09-10T10:01:50.321297+0000","last_clean":"2024-09-10T10:01:50.321297+0000","last_became_active":"2024-09-10T10:01:34.253395+0000","last_became_peered":"2024-09-10T10:01:34.253395+0000","last_unstale":"2024-09-10T10:01:50.321297+0000","last_undegraded":"2024-09-10T10:01:50.321297+0000","last_fullsized":"2024-09-10T10:01:50.321297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:55.264734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337074+0000","last_change":"2024-09-10T10:01:50.337074+0000","last_active":"2024-09-10T10:01:50.337074+0000","last_peered":"2024-09-10T10:01:50.337074+0000","last_clean":"2024-09-10T10:01:50.337074+0000","last_became_active":"2024-09-10T10:01:50.336661+0000","last_became_peered":"2024-09-10T10:01:50.336661+0000","last_unstale":"2024-09-10T10:01:50.337074+0000","last_undegraded":"2024-09-10T10:01:50.337074+0000","last_fullsized":"2024-09-10T10:01:50.337074+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:47.700516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338318+0000","last_change":"2024-09-10T10:01:34.249342+0000","last_active":"2024-09-10T10:01:50.338318+0000","last_peered":"2024-09-10T10:01:50.338318+0000","last_clean":"2024-09-10T10:01:50.338318+0000","last_became_active":"2024-09-10T10:01:34.245289+0000","last_became_peered":"2024-09-10T10:01:34.245289+0000","last_unstale":"2024-09-10T10:01:50.338318+0000","last_undegraded":"2024-09-10T10:01:50.338318+0000","last_fullsized":"2024-09-10T10:01:50.338318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:39:34.068515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320070+0000","last_change":"2024-09-10T10:01:32.199112+0000","last_active":"2024-09-10T10:01:50.320070+0000","last_peered":"2024-09-10T10:01:50.320070+0000","last_clean":"2024-09-10T10:01:50.320070+0000","last_became_active":"2024-09-10T10:01:32.198627+0000","last_became_peered":"2024-09-10T10:01:32.198627+0000","last_unstale":"2024-09-10T10:01:50.320070+0000","last_undegraded":"2024-09-10T10:01:50.320070+0000","last_fullsized":"2024-09-10T10:01:50.320070+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:20:37.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.335789+0000","last_change":"2024-09-10T10:01:34.253537+0000","last_active":"2024-09-10T10:01:50.335789+0000","last_peered":"2024-09-10T10:01:50.335789+0000","last_clean":"2024-09-10T10:01:50.335789+0000","last_became_active":"2024-09-10T10:01:34.253060+0000","last_became_peered":"2024-09-10T10:01:34.253060+0000","last_unstale":"2024-09-10T10:01:50.335789+0000","last_undegraded":"2024-09-10T10:01:50.335789+0000","last_fullsized":"2024-09-10T10:01:50.335789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:17.411289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.333010+0000","last_change":"2024-09-10T10:01:32.202595+0000","last_active":"2024-09-10T10:01:50.333010+0000","last_peered":"2024-09-10T10:01:50.333010+0000","last_clean":"2024-09-10T10:01:50.333010+0000","last_became_active":"2024-09-10T10:01:32.201796+0000","last_became_peered":"2024-09-10T10:01:32.201796+0000","last_unstale":"2024-09-10T10:01:50.333010+0000","last_undegraded":"2024-09-10T10:01:50.333010+0000","last_fullsized":"2024-09-10T10:01:50.333010+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:54.778442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320621+0000","last_change":"2024-09-10T10:01:34.244734+0000","last_active":"2024-09-10T10:01:50.320621+0000","last_peered":"2024-09-10T10:01:50.320621+0000","last_clean":"2024-09-10T10:01:50.320621+0000","last_became_active":"2024-09-10T10:01:34.244396+0000","last_became_peered":"2024-09-10T10:01:34.244396+0000","last_unstale":"2024-09-10T10:01:50.320621+0000","last_undegraded":"2024-09-10T10:01:50.320621+0000","last_fullsized":"2024-09-10T10:01:50.320621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:02.266670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338190+0000","last_change":"2024-09-10T10:01:32.207322+0000","last_active":"2024-09-10T10:01:50.338190+0000","last_peered":"2024-09-10T10:01:50.338190+0000","last_clean":"2024-09-10T10:01:50.338190+0000","last_became_active":"2024-09-10T10:01:32.206936+0000","last_became_peered":"2024-09-10T10:01:32.206936+0000","last_unstale":"2024-09-10T10:01:50.338190+0000","last_undegraded":"2024-09-10T10:01:50.338190+0000","last_fullsized":"2024-09-10T10:01:50.338190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:17.194749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.337740+0000","last_change":"2024-09-10T10:01:34.255626+0000","last_active":"2024-09-10T10:01:50.337740+0000","last_peered":"2024-09-10T10:01:50.337740+0000","last_clean":"2024-09-10T10:01:50.337740+0000","last_became_active":"2024-09-10T10:01:34.255333+0000","last_became_peered":"2024-09-10T10:01:34.255333+0000","last_unstale":"2024-09-10T10:01:50.337740+0000","last_undegraded":"2024-09-10T10:01:50.337740+0000","last_fullsized":"2024-09-10T10:01:50.337740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:26.287576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.710505+0000","last_change":"2024-09-10T10:01:34.257209+0000","last_active":"2024-09-10T10:01:50.710505+0000","last_peered":"2024-09-10T10:01:50.710505+0000","last_clean":"2024-09-10T10:01:50.710505+0000","last_became_active":"2024-09-10T10:01:34.256891+0000","last_became_peered":"2024-09-10T10:01:34.256891+0000","last_unstale":"2024-09-10T10:01:50.710505+0000","last_undegraded":"2024-09-10T10:01:50.710505+0000","last_fullsized":"2024-09-10T10:01:50.710505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:58.178876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.338664+0000","last_change":"2024-09-10T10:01:32.200462+0000","last_active":"2024-09-10T10:01:50.338664+0000","last_peered":"2024-09-10T10:01:50.338664+0000","last_clean":"2024-09-10T10:01:50.338664+0000","last_became_active":"2024-09-10T10:01:32.198970+0000","last_became_peered":"2024-09-10T10:01:32.198970+0000","last_unstale":"2024-09-10T10:01:50.338664+0000","last_undegraded":"2024-09-10T10:01:50.338664+0000","last_fullsized":"2024-09-10T10:01:50.338664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:00.980894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.318326+0000","last_change":"2024-09-10T10:01:34.246984+0000","last_active":"2024-09-10T10:01:50.318326+0000","last_peered":"2024-09-10T10:01:50.318326+0000","last_clean":"2024-09-10T10:01:50.318326+0000","last_became_active":"2024-09-10T10:01:34.246244+0000","last_became_peered":"2024-09-10T10:01:34.246244+0000","last_unstale":"2024-09-10T10:01:50.318326+0000","last_undegraded":"2024-09-10T10:01:50.318326+0000","last_fullsized":"2024-09-10T10:01:50.318326+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:47.656959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'2","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.342048+0000","last_change":"2024-09-10T10:01:50.341594+0000","last_active":"2024-09-10T10:01:50.342048+0000","last_peered":"2024-09-10T10:01:50.342048+0000","last_clean":"2024-09-10T10:01:50.342048+0000","last_became_active":"2024-09-10T10:01:50.338152+0000","last_became_peered":"2024-09-10T10:01:50.338152+0000","last_unstale":"2024-09-10T10:01:50.342048+0000","last_undegraded":"2024-09-10T10:01:50.342048+0000","last_fullsized":"2024-09-10T10:01:50.342048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:25.484321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320803+0000","last_change":"2024-09-10T10:01:34.246396+0000","last_active":"2024-09-10T10:01:50.320803+0000","last_peered":"2024-09-10T10:01:50.320803+0000","last_clean":"2024-09-10T10:01:50.320803+0000","last_became_active":"2024-09-10T10:01:34.245956+0000","last_became_peered":"2024-09-10T10:01:34.245956+0000","last_unstale":"2024-09-10T10:01:50.320803+0000","last_undegraded":"2024-09-10T10:01:50.320803+0000","last_fullsized":"2024-09-10T10:01:50.320803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:53.440984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320437+0000","last_change":"2024-09-10T10:01:32.529907+0000","last_active":"2024-09-10T10:01:50.320437+0000","last_peered":"2024-09-10T10:01:50.320437+0000","last_clean":"2024-09-10T10:01:50.320437+0000","last_became_active":"2024-09-10T10:01:32.529353+0000","last_became_peered":"2024-09-10T10:01:32.529353+0000","last_unstale":"2024-09-10T10:01:50.320437+0000","last_undegraded":"2024-09-10T10:01:50.320437+0000","last_fullsized":"2024-09-10T10:01:50.320437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:45.562270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.320201+0000","last_change":"2024-09-10T10:01:34.243623+0000","last_active":"2024-09-10T10:01:50.320201+0000","last_peered":"2024-09-10T10:01:50.320201+0000","last_clean":"2024-09-10T10:01:50.320201+0000","last_became_active":"2024-09-10T10:01:34.241625+0000","last_became_peered":"2024-09-10T10:01:34.241625+0000","last_unstale":"2024-09-10T10:01:50.320201+0000","last_undegraded":"2024-09-10T10:01:50.320201+0000","last_fullsized":"2024-09-10T10:01:50.320201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:33.219879+0000","last_clean_scrub_stamp":"2024-09-10T10:01:33.219879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:25.112837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:01:50.317895+0000","last_change":"2024-09-10T10:01:32.205514+0000","last_active":"2024-09-10T10:01:50.317895+0000","last_peered":"2024-09-10T10:01:50.317895+0000","last_clean":"2024-09-10T10:01:50.317895+0000","last_became_active":"2024-09-10T10:01:32.205127+0000","last_became_peered":"2024-09-10T10:01:32.205127+0000","last_unstale":"2024-09-10T10:01:50.317895+0000","last_undegraded":"2024-09-10T10:01:50.317895+0000","last_fullsized":"2024-09-10T10:01:50.317895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:01:31.184965+0000","last_clean_scrub_stamp":"2024-09-10T10:01:31.184965+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:54.207240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63617,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":115,"num_read_kb":121,"num_write":83,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":190851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":892928,"data_stored":437776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1091274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939682,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":438380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40968,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704184,"statfs":{"total":95995035648,"available":95953084416,"internally_reserved":0,"allocated":1495040,"data_stored":1028843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1503232,"data_stored":1028443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":897024,"data_stored":437798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":437997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":500629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":500595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":437866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":63013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:02:38.331 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T10:02:38.332 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T10:02:38.332 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T10:02:38.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T10:02:38.592 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:38.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:38 smithi045 ceph-mon[31114]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:38.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:38 smithi162 ceph-mon[34963]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:38.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:38 smithi047 ceph-mon[35244]: pgmap v307: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.053+0000 7fc52300c640 1 -- 172.21.15.45:0/1561478418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c106f10 msgr2=0x7fc51c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.053+0000 7fc52300c640 1 --2- 172.21.15.45:0/1561478418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c106f10 0x7fc51c10ea90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc514009ca0 tx=0x7fc51402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 -- 172.21.15.45:0/1561478418 shutdown_connections 2024-09-10T10:02:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 --2- 172.21.15.45:0/1561478418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c106f10 0x7fc51c10ea90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 --2- 172.21.15.45:0/1561478418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc51c102790 0x7fc51c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 --2- 172.21.15.45:0/1561478418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc51c101de0 0x7fc51c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 -- 172.21.15.45:0/1561478418 >> 172.21.15.45:0/1561478418 conn(0x7fc51c0fd8c0 msgr2=0x7fc51c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 -- 172.21.15.45:0/1561478418 shutdown_connections 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 -- 172.21.15.45:0/1561478418 wait complete. 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 Processor -- start 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 -- start start 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc51c101de0 0x7fc51c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c102790 0x7fc51c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.054+0000 7fc52300c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc51c106f10 0x7fc51c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc52300c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c111f90 con 0x7fc51c102790 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc52300c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c111e10 con 0x7fc51c101de0 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc52300c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c112110 con 0x7fc51c106f10 2024-09-10T10:02:39.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c102790 0x7fc51c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c102790 0x7fc51c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59134/0 (socket says 172.21.15.45:59134) 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 -- 172.21.15.45:0/1336314928 learned_addr learned my addr 172.21.15.45:0/1336314928 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc520da8640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc51c101de0 0x7fc51c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc5215a9640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc51c106f10 0x7fc51c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 -- 172.21.15.45:0/1336314928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc51c106f10 msgr2=0x7fc51c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc51c106f10 0x7fc51c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 -- 172.21.15.45:0/1336314928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc51c101de0 msgr2=0x7fc51c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc51c101de0 0x7fc51c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 -- 172.21.15.45:0/1336314928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c1a6920 con 0x7fc51c102790 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc51bfff640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c102790 0x7fc51c074a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc50400c890 tx=0x7fc50400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc519ffb640 1 -- 172.21.15.45:0/1336314928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504004220 con 0x7fc51c102790 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc519ffb640 1 -- 172.21.15.45:0/1336314928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5040043c0 con 0x7fc51c102790 2024-09-10T10:02:39.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c1a6c10 con 0x7fc51c102790 2024-09-10T10:02:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.055+0000 7fc519ffb640 1 -- 172.21.15.45:0/1336314928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504004a00 con 0x7fc51c102790 2024-09-10T10:02:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.056+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c1a71a0 con 0x7fc51c102790 2024-09-10T10:02:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.057+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51c1021c0 con 0x7fc51c102790 2024-09-10T10:02:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.058+0000 7fc519ffb640 1 -- 172.21.15.45:0/1336314928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc5040079b0 con 0x7fc51c102790 2024-09-10T10:02:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.059+0000 7fc519ffb640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc5080a2c90 0x7fc5080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:39.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.059+0000 7fc520da8640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc5080a2c90 0x7fc5080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.059+0000 7fc519ffb640 1 -- 172.21.15.45:0/1336314928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc5040cee10 con 0x7fc51c102790 2024-09-10T10:02:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.060+0000 7fc520da8640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc5080a2c90 0x7fc5080a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009990 tx=0x7fc50c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:39.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.063+0000 7fc519ffb640 1 -- 172.21.15.45:0/1336314928 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc504014030 con 0x7fc51c102790 2024-09-10T10:02:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.275+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc51c10a0f0 con 0x7fc51c102790 2024-09-10T10:02:39.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.276+0000 7fc519ffb640 1 -- 172.21.15.45:0/1336314928 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc50409bc20 con 0x7fc51c102790 2024-09-10T10:02:39.277 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:39.277 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T10:02:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.278+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc5080a2c90 msgr2=0x7fc5080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc5080a2c90 0x7fc5080a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009990 tx=0x7fc50c009110 comp rx=0 tx=0).stop 2024-09-10T10:02:39.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c102790 msgr2=0x7fc51c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c102790 0x7fc51c074a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc50400c890 tx=0x7fc50400cc90 comp rx=0 tx=0).stop 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 shutdown_connections 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fc5080a2c90 0x7fc5080a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc51c106f10 0x7fc51c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc51c102790 0x7fc51c074a30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 --2- 172.21.15.45:0/1336314928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc51c101de0 0x7fc51c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 >> 172.21.15.45:0/1336314928 conn(0x7fc51c0fd8c0 msgr2=0x7fc51c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 shutdown_connections 2024-09-10T10:02:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:39.279+0000 7fc52300c640 1 -- 172.21.15.45:0/1336314928 wait complete. 2024-09-10T10:02:39.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:39 smithi045 ceph-mon[31114]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:39.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:39 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1336314928' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:02:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:39 smithi162 ceph-mon[34963]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:39 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1336314928' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:02:39.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:39 smithi047 ceph-mon[35244]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T10:02:39.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:39 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1336314928' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T10:02:39.986 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T10:02:39.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:40.247 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:40.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:40 smithi045 ceph-mon[31114]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:40.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 -- 172.21.15.45:0/2951752778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164102790 msgr2=0x7f21641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 --2- 172.21.15.45:0/2951752778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164102790 0x7f21641067e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2150009900 tx=0x7f215002a410 comp rx=0 tx=0).stop 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 -- 172.21.15.45:0/2951752778 shutdown_connections 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 --2- 172.21.15.45:0/2951752778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2164106f10 0x7f216410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 --2- 172.21.15.45:0/2951752778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164102790 0x7f21641067e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 --2- 172.21.15.45:0/2951752778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2164101de0 0x7f21641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 -- 172.21.15.45:0/2951752778 >> 172.21.15.45:0/2951752778 conn(0x7f21640fd8c0 msgr2=0x7f21640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 -- 172.21.15.45:0/2951752778 shutdown_connections 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.661+0000 7f216b1c4640 1 -- 172.21.15.45:0/2951752778 wait complete. 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 Processor -- start 2024-09-10T10:02:40.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 -- start start 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2164101de0 0x7f21640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2164102790 0x7f2164074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164106f10 0x7f2164074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2164111f90 con 0x7f2164106f10 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2164111e10 con 0x7f2164101de0 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f216b1c4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2164112110 con 0x7f2164102790 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f2169761640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164106f10 0x7f2164074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f2169761640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164106f10 0x7f2164074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59142/0 (socket says 172.21.15.45:59142) 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f2169761640 1 -- 172.21.15.45:0/654383719 learned_addr learned my addr 172.21.15.45:0/654383719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f2163fff640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2164102790 0x7f2164074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.662+0000 7f2168f60640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2164101de0 0x7f21640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2169761640 1 -- 172.21.15.45:0/654383719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2164102790 msgr2=0x7f2164074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2169761640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2164102790 0x7f2164074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2169761640 1 -- 172.21.15.45:0/654383719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2164101de0 msgr2=0x7f21640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2169761640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2164101de0 0x7f21640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2169761640 1 -- 172.21.15.45:0/654383719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21641a69e0 con 0x7f2164106f10 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2168f60640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2164101de0 0x7f21640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2169761640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164106f10 0x7f2164074f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f215c00b970 tx=0x7f215c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2161ffb640 1 -- 172.21.15.45:0/654383719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c019070 con 0x7f2164106f10 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21641a6cd0 con 0x7f2164106f10 2024-09-10T10:02:40.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2161ffb640 1 -- 172.21.15.45:0/654383719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215c00cb40 con 0x7f2164106f10 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f2161ffb640 1 -- 172.21.15.45:0/654383719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c007610 con 0x7f2164106f10 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.663+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21641a71d0 con 0x7f2164106f10 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.664+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21641021c0 con 0x7f2164106f10 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.665+0000 7f2161ffb640 1 -- 172.21.15.45:0/654383719 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f215c01c070 con 0x7f2164106f10 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.666+0000 7f2161ffb640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f21540a2c10 0x7f21540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.666+0000 7f2168f60640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f21540a2c10 0x7f21540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.666+0000 7f2161ffb640 1 -- 172.21.15.45:0/654383719 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f215c0ce620 con 0x7f2164106f10 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.667+0000 7f2168f60640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f21540a2c10 0x7f21540a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f21580058c0 tx=0x7f2158005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.670+0000 7f2161ffb640 1 -- 172.21.15.45:0/654383719 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f215c09aef0 con 0x7f2164106f10 2024-09-10T10:02:40.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.884+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f216410a150 con 0x7f2164106f10 2024-09-10T10:02:40.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.885+0000 7f2161ffb640 1 -- 172.21.15.45:0/654383719 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11843 (secure 0 0 0) 0x7f215c09f1f0 con 0x7f2164106f10 2024-09-10T10:02:40.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:40.887 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"btime":"2024-09-10T10:01:44:271138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:01:43.932833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:02:40.887 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-09-10T10:02:40.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f21540a2c10 msgr2=0x7f21540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f21540a2c10 0x7f21540a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f21580058c0 tx=0x7f2158005ec0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164106f10 msgr2=0x7f2164074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164106f10 0x7f2164074f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f215c00b970 tx=0x7f215c00be30 comp rx=0 tx=0).stop 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 shutdown_connections 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f21540a2c10 0x7f21540a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2164106f10 0x7f2164074f70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2164102790 0x7f2164074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 --2- 172.21.15.45:0/654383719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2164101de0 0x7f21640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 >> 172.21.15.45:0/654383719 conn(0x7f21640fd8c0 msgr2=0x7f21640fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 shutdown_connections 2024-09-10T10:02:40.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:40.887+0000 7f216b1c4640 1 -- 172.21.15.45:0/654383719 wait complete. 2024-09-10T10:02:40.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:40 smithi162 ceph-mon[34963]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:40.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:40 smithi047 ceph-mon[35244]: pgmap v308: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:41.467 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:41.722 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:41.843 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:41 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/654383719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:41.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:41 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/654383719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:41.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:41 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/654383719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 -- 172.21.15.45:0/2226812595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c10c170 msgr2=0x7f942c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 --2- 172.21.15.45:0/2226812595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c10c170 0x7f942c10e610 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9424009ca0 tx=0x7f942402a950 comp rx=0 tx=0).stop 2024-09-10T10:02:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 -- 172.21.15.45:0/2226812595 shutdown_connections 2024-09-10T10:02:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 --2- 172.21.15.45:0/2226812595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c10c170 0x7f942c10e610 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 --2- 172.21.15.45:0/2226812595 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f942c103660 0x7f942c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 --2- 172.21.15.45:0/2226812595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c102d40 0x7f942c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 -- 172.21.15.45:0/2226812595 >> 172.21.15.45:0/2226812595 conn(0x7f942c0fc7a0 msgr2=0x7f942c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 -- 172.21.15.45:0/2226812595 shutdown_connections 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.194+0000 7f94310e4640 1 -- 172.21.15.45:0/2226812595 wait complete. 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.195+0000 7f94310e4640 1 Processor -- start 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.195+0000 7f94310e4640 1 -- start start 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f94310e4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f942c102d40 0x7f942c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f94310e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c103660 0x7f942c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f94310e4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c10c170 0x7f942c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f94310e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f942c111f50 con 0x7f942c103660 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f94310e4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f942c111dd0 con 0x7f942c10c170 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f94310e4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f942c1120d0 con 0x7f942c102d40 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c103660 0x7f942c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c103660 0x7f942c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59160/0 (socket says 172.21.15.45:59160) 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 -- 172.21.15.45:0/3724721325 learned_addr learned my addr 172.21.15.45:0/3724721325 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942ad9d640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f942c102d40 0x7f942c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942b59e640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c10c170 0x7f942c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 -- 172.21.15.45:0/3724721325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f942c102d40 msgr2=0x7f942c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f942c102d40 0x7f942c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 -- 172.21.15.45:0/3724721325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c10c170 msgr2=0x7f942c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c10c170 0x7f942c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 -- 172.21.15.45:0/3724721325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c19d410 con 0x7f942c103660 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942b59e640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c10c170 0x7f942c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f942a59c640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c103660 0x7f942c1989a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f942000d750 tx=0x7f942000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f9413fff640 1 -- 172.21.15.45:0/3724721325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420017070 con 0x7f942c103660 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f9413fff640 1 -- 172.21.15.45:0/3724721325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9420005020 con 0x7f942c103660 2024-09-10T10:02:42.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.196+0000 7f9413fff640 1 -- 172.21.15.45:0/3724721325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420004d00 con 0x7f942c103660 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.197+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c19d700 con 0x7f942c103660 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.197+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c1a4fe0 con 0x7f942c103660 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.198+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c103120 con 0x7f942c103660 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.199+0000 7f9413fff640 1 -- 172.21.15.45:0/3724721325 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9420009a30 con 0x7f942c103660 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.199+0000 7f9413fff640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94140a2c90 0x7f94140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.200+0000 7f942ad9d640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94140a2c90 0x7f94140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.200+0000 7f9413fff640 1 -- 172.21.15.45:0/3724721325 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f94200cef00 con 0x7f942c103660 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.200+0000 7f942ad9d640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94140a2c90 0x7f94140a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f941c005d00 tx=0x7f941c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:42.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.203+0000 7f9413fff640 1 -- 172.21.15.45:0/3724721325 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f942009bd90 con 0x7f942c103660 2024-09-10T10:02:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.414+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f942c1997c0 con 0x7f942c103660 2024-09-10T10:02:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.415+0000 7f9413fff640 1 -- 172.21.15.45:0/3724721325 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11843 (secure 0 0 0) 0x7f942c1997c0 con 0x7f942c103660 2024-09-10T10:02:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:42.417 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"btime":"2024-09-10T10:01:44:271138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:01:43.932833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:02:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-09-10T10:02:42.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94140a2c90 msgr2=0x7f94140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94140a2c90 0x7f94140a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f941c005d00 tx=0x7f941c00b410 comp rx=0 tx=0).stop 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c103660 msgr2=0x7f942c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c103660 0x7f942c1989a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f942000d750 tx=0x7f942000dc10 comp rx=0 tx=0).stop 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 shutdown_connections 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94140a2c90 0x7f94140a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f942c10c170 0x7f942c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f942c103660 0x7f942c1989a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 --2- 172.21.15.45:0/3724721325 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f942c102d40 0x7f942c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 >> 172.21.15.45:0/3724721325 conn(0x7f942c0fc7a0 msgr2=0x7f942c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 shutdown_connections 2024-09-10T10:02:42.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:42.417+0000 7f94310e4640 1 -- 172.21.15.45:0/3724721325 wait complete. 2024-09-10T10:02:42.750 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:42 smithi045 ceph-mon[31114]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:42.751 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:42 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3724721325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:42.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:42 smithi162 ceph-mon[34963]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:42.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:42 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3724721325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:42.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:42 smithi047 ceph-mon[35244]: pgmap v309: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:42.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:42 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3724721325' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:43.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:02:43.308 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.753+0000 7fb571ba0640 1 -- 172.21.15.45:0/145381700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c101de0 msgr2=0x7fb56c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.753+0000 7fb571ba0640 1 --2- 172.21.15.45:0/145381700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c101de0 0x7fb56c1021c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009900 tx=0x7fb55c02a410 comp rx=0 tx=0).stop 2024-09-10T10:02:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.753+0000 7fb571ba0640 1 -- 172.21.15.45:0/145381700 shutdown_connections 2024-09-10T10:02:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.753+0000 7fb571ba0640 1 --2- 172.21.15.45:0/145381700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb56c106f10 0x7fb56c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.753+0000 7fb571ba0640 1 --2- 172.21.15.45:0/145381700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb56c102790 0x7fb56c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.753+0000 7fb571ba0640 1 --2- 172.21.15.45:0/145381700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c101de0 0x7fb56c1021c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.753+0000 7fb571ba0640 1 -- 172.21.15.45:0/145381700 >> 172.21.15.45:0/145381700 conn(0x7fb56c0fd8c0 msgr2=0x7fb56c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.754+0000 7fb571ba0640 1 -- 172.21.15.45:0/145381700 shutdown_connections 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.754+0000 7fb571ba0640 1 -- 172.21.15.45:0/145381700 wait complete. 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.754+0000 7fb571ba0640 1 Processor -- start 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb571ba0640 1 -- start start 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb571ba0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb56c101de0 0x7fb56c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb571ba0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb56c102790 0x7fb56c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb571ba0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c106f10 0x7fb56c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb571ba0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c111f90 con 0x7fb56c106f10 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb571ba0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c111e10 con 0x7fb56c101de0 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb571ba0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c112110 con 0x7fb56c102790 2024-09-10T10:02:43.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c106f10 0x7fb56c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c106f10 0x7fb56c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33814/0 (socket says 172.21.15.45:33814) 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 -- 172.21.15.45:0/2462127638 learned_addr learned my addr 172.21.15.45:0/2462127638 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56b7fe640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb56c101de0 0x7fb56c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56affd640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb56c102790 0x7fb56c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 -- 172.21.15.45:0/2462127638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb56c102790 msgr2=0x7fb56c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb56c102790 0x7fb56c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 -- 172.21.15.45:0/2462127638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb56c101de0 msgr2=0x7fb56c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb56c101de0 0x7fb56c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.755+0000 7fb56bfff640 1 -- 172.21.15.45:0/2462127638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c19d440 con 0x7fb56c106f10 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.756+0000 7fb56bfff640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c106f10 0x7fb56c19cd60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb56400b970 tx=0x7fb56400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.756+0000 7fb568ff9640 1 -- 172.21.15.45:0/2462127638 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564019070 con 0x7fb56c106f10 2024-09-10T10:02:43.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.756+0000 7fb568ff9640 1 -- 172.21.15.45:0/2462127638 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56400cb40 con 0x7fb56c106f10 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.756+0000 7fb568ff9640 1 -- 172.21.15.45:0/2462127638 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564007650 con 0x7fb56c106f10 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.756+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56c19d730 con 0x7fb56c106f10 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.756+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb56c1a5010 con 0x7fb56c106f10 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.757+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56c1021c0 con 0x7fb56c106f10 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.758+0000 7fb568ff9640 1 -- 172.21.15.45:0/2462127638 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb56401c070 con 0x7fb56c106f10 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.759+0000 7fb568ff9640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb5540a2c10 0x7fb5540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.759+0000 7fb56b7fe640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb5540a2c10 0x7fb5540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.759+0000 7fb568ff9640 1 -- 172.21.15.45:0/2462127638 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb5640ceb50 con 0x7fb56c106f10 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.759+0000 7fb56b7fe640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb5540a2c10 0x7fb5540a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb55c000c00 tx=0x7fb55c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:43.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.762+0000 7fb568ff9640 1 -- 172.21.15.45:0/2462127638 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb56409b420 con 0x7fb56c106f10 2024-09-10T10:02:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.937+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb56c109b60 con 0x7fb56c106f10 2024-09-10T10:02:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.939+0000 7fb568ff9640 1 -- 172.21.15.45:0/2462127638 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22675 (secure 0 0 0) 0x7fb56409f720 con 0x7fb56c106f10 2024-09-10T10:02:43.940 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:43.941 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:01:50.297428+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb5540a2c10 msgr2=0x7fb5540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb5540a2c10 0x7fb5540a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb55c000c00 tx=0x7fb55c0058d0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c106f10 msgr2=0x7fb56c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c106f10 0x7fb56c19cd60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb56400b970 tx=0x7fb56400be30 comp rx=0 tx=0).stop 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 shutdown_connections 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fb5540a2c10 0x7fb5540a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb56c106f10 0x7fb56c19cd60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb56c102790 0x7fb56c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 --2- 172.21.15.45:0/2462127638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb56c101de0 0x7fb56c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 >> 172.21.15.45:0/2462127638 conn(0x7fb56c0fd8c0 msgr2=0x7fb56c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:43.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 shutdown_connections 2024-09-10T10:02:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:43.941+0000 7fb571ba0640 1 -- 172.21.15.45:0/2462127638 wait complete. 2024-09-10T10:02:44.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:44.886 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:44 smithi162 ceph-mon[34963]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:44.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:44 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2462127638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:44.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:44 smithi045 ceph-mon[31114]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:44.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:44 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2462127638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:44.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:44 smithi047 ceph-mon[35244]: pgmap v310: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:44.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:44 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2462127638' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:02:45.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 -- 172.21.15.45:0/1787484395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341036e0 msgr2=0x7f7334107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 --2- 172.21.15.45:0/1787484395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341036e0 0x7f7334107b90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f731c009930 tx=0x7f731c02a420 comp rx=0 tx=0).stop 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 -- 172.21.15.45:0/1787484395 shutdown_connections 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 --2- 172.21.15.45:0/1787484395 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f73341082a0 0x7f733410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 --2- 172.21.15.45:0/1787484395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341036e0 0x7f7334107b90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 --2- 172.21.15.45:0/1787484395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7334102d30 0x7f7334103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 -- 172.21.15.45:0/1787484395 >> 172.21.15.45:0/1787484395 conn(0x7f73340fc7d0 msgr2=0x7f73340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 -- 172.21.15.45:0/1787484395 shutdown_connections 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.326+0000 7f733940b640 1 -- 172.21.15.45:0/1787484395 wait complete. 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 Processor -- start 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 -- start start 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7334102d30 0x7f7334100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73341036e0 0x7f7334100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341082a0 0x7f7334101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73341120c0 con 0x7f73341082a0 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7334111f40 con 0x7f73341036e0 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f733940b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7334112240 con 0x7f7334102d30 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341082a0 0x7f7334101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f732bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73341036e0 0x7f7334100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341082a0 0x7f7334101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33838/0 (socket says 172.21.15.45:33838) 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 -- 172.21.15.45:0/3398975588 learned_addr learned my addr 172.21.15.45:0/3398975588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f7332ffd640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7334102d30 0x7f7334100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 -- 172.21.15.45:0/3398975588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7334102d30 msgr2=0x7f7334100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7334102d30 0x7f7334100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 -- 172.21.15.45:0/3398975588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73341036e0 msgr2=0x7f7334100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73341036e0 0x7f7334100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.327+0000 7f73337fe640 1 -- 172.21.15.45:0/3398975588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73341a6ba0 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.328+0000 7f73337fe640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341082a0 0x7f7334101410 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f732c00b970 tx=0x7f732c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.328+0000 7f7330ff9640 1 -- 172.21.15.45:0/3398975588 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c019070 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.328+0000 7f7330ff9640 1 -- 172.21.15.45:0/3398975588 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732c00cb40 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.328+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73341a6e90 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.328+0000 7f7330ff9640 1 -- 172.21.15.45:0/3398975588 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c007650 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.328+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73341a7340 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.329+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7334103110 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.331+0000 7f7330ff9640 1 -- 172.21.15.45:0/3398975588 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f732c01c070 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.332+0000 7f7330ff9640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f73200a2c90 0x7f73200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.332+0000 7f7332ffd640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f73200a2c90 0x7f73200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.332+0000 7f7330ff9640 1 -- 172.21.15.45:0/3398975588 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f732c0cea40 con 0x7f73341082a0 2024-09-10T10:02:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.332+0000 7f7332ffd640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f73200a2c90 0x7f73200a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73240058c0 tx=0x7f732400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.335+0000 7f7330ff9640 1 -- 172.21.15.45:0/3398975588 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f732c09b2b0 con 0x7f73341082a0 2024-09-10T10:02:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.547+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7334101ec0 con 0x7f73341082a0 2024-09-10T10:02:45.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.548+0000 7f7330ff9640 1 -- 172.21.15.45:0/3398975588 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11843 (secure 0 0 0) 0x7f732c095e20 con 0x7f73341082a0 2024-09-10T10:02:45.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:45.554 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"btime":"2024-09-10T10:01:44:271138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:01:43.932833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:02:45.554 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-09-10T10:02:45.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.550+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f73200a2c90 msgr2=0x7f73200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.550+0000 7f733940b640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f73200a2c90 0x7f73200a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73240058c0 tx=0x7f732400a2d0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.550+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341082a0 msgr2=0x7f7334101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.550+0000 7f733940b640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341082a0 0x7f7334101410 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f732c00b970 tx=0x7f732c00be30 comp rx=0 tx=0).stop 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 shutdown_connections 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f73200a2c90 0x7f73200a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73341082a0 0x7f7334101410 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f73341036e0 0x7f7334100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 --2- 172.21.15.45:0/3398975588 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7334102d30 0x7f7334100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 >> 172.21.15.45:0/3398975588 conn(0x7f73340fc7d0 msgr2=0x7f7334106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 shutdown_connections 2024-09-10T10:02:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:45.551+0000 7f733940b640 1 -- 172.21.15.45:0/3398975588 wait complete. 2024-09-10T10:02:46.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T10:01:34.924889+0000', 'modified': '2024-09-10T10:01:43.932833+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6839/2065513838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 2065513838}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 2065513838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6839/2511396298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6838', 'nonce': 2511396298}, {'type': 'v1', 'addr': '172.21.15.47:6839', 'nonce': 2511396298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24488]} 2024-09-10T10:02:46.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:02:46.217 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T10:02:46.477 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:46.696 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:46 smithi045 ceph-mon[31114]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:46.696 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:46 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3398975588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:46.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:46 smithi162 ceph-mon[34963]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:46.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:46 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3398975588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:46.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:46 smithi047 ceph-mon[35244]: pgmap v311: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:46.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:46 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3398975588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:46.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 -- 172.21.15.45:0/1737761560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420102790 msgr2=0x7f64201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 --2- 172.21.15.45:0/1737761560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420102790 0x7f64201067e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6414009900 tx=0x7f641402a410 comp rx=0 tx=0).stop 2024-09-10T10:02:46.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 -- 172.21.15.45:0/1737761560 shutdown_connections 2024-09-10T10:02:46.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 --2- 172.21.15.45:0/1737761560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6420106f10 0x7f642010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 --2- 172.21.15.45:0/1737761560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420102790 0x7f64201067e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 --2- 172.21.15.45:0/1737761560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6420101de0 0x7f64201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 -- 172.21.15.45:0/1737761560 >> 172.21.15.45:0/1737761560 conn(0x7f64200fd8c0 msgr2=0x7f64200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 -- 172.21.15.45:0/1737761560 shutdown_connections 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.909+0000 7f642632e640 1 -- 172.21.15.45:0/1737761560 wait complete. 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 Processor -- start 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 -- start start 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6420101de0 0x7f6420198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6420102790 0x7f6420198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420106f10 0x7f642019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6420111f80 con 0x7f6420106f10 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6420111e00 con 0x7f6420102790 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f642632e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6420112100 con 0x7f6420101de0 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f64248cb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420106f10 0x7f642019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f64248cb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420106f10 0x7f642019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33844/0 (socket says 172.21.15.45:33844) 2024-09-10T10:02:46.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.910+0000 7f64248cb640 1 -- 172.21.15.45:0/8503683 learned_addr learned my addr 172.21.15.45:0/8503683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f641ffff640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6420101de0 0x7f6420198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f641f7fe640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6420102790 0x7f6420198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f64248cb640 1 -- 172.21.15.45:0/8503683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6420101de0 msgr2=0x7f6420198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f64248cb640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6420101de0 0x7f6420198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f64248cb640 1 -- 172.21.15.45:0/8503683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6420102790 msgr2=0x7f6420198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f64248cb640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6420102790 0x7f6420198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f64248cb640 1 -- 172.21.15.45:0/8503683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642019d300 con 0x7f6420106f10 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f641f7fe640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6420102790 0x7f6420198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f64248cb640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420106f10 0x7f642019cc20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f641800eaa0 tx=0x7f641800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f641d7fa640 1 -- 172.21.15.45:0/8503683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418004d20 con 0x7f6420106f10 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f641d7fa640 1 -- 172.21.15.45:0/8503683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6418005020 con 0x7f6420106f10 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642019d5f0 con 0x7f6420106f10 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f641d7fa640 1 -- 172.21.15.45:0/8503683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180116d0 con 0x7f6420106f10 2024-09-10T10:02:46.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.911+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64201a4ed0 con 0x7f6420106f10 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.913+0000 7f641d7fa640 1 -- 172.21.15.45:0/8503683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f64180051d0 con 0x7f6420106f10 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 Processor -- start 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 -- start start 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639b99d0 0x5575639b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639c4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557563920090 con 0x5575639ba2f0 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5575638c7a70 con 0x5575639b99d0 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f6428707740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55756391c050 con 0x5575639c4bb0 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641d7fa640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64080a2c10 0x7f64080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33846/0 (socket says 172.21.15.45:33846) 2024-09-10T10:02:46.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641f7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639b99d0 0x5575639b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 -- 172.21.15.45:0/3610704801 learned_addr learned my addr 172.21.15.45:0/3610704801 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f64248cb640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64080a2c10 0x7f64080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641f7fe640 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 -- 172.21.15.45:0/3610704801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 msgr2=0x5575639c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 -- 172.21.15.45:0/3610704801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639b99d0 msgr2=0x5575639b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639b99d0 0x5575639b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641ffff640 1 -- 172.21.15.45:0/3610704801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575638d4130 con 0x5575639ba2f0 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.914+0000 7f641f7fe640 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.915+0000 7f641d7fa640 1 -- 172.21.15.45:0/8503683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f64180cf380 con 0x7f6420106f10 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.915+0000 7f641ffff640 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639c4670 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f641000ad00 tx=0x7f641000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b0050a957932e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.915+0000 7f64248cb640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64080a2c10 0x7f64080a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f641809ba20 tx=0x7f6418097630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:46.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.915+0000 7f64067fc640 1 -- 172.21.15.45:0/3610704801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641001d070 con 0x5575639ba2f0 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.915+0000 7f64067fc640 1 -- 172.21.15.45:0/3610704801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6410018b20 con 0x5575639ba2f0 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.915+0000 7f64067fc640 1 -- 172.21.15.45:0/3610704801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410010bc0 con 0x5575639ba2f0 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 -- 172.21.15.45:0/3610704801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 msgr2=0x5575639c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639c4670 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f641000ad00 tx=0x7f641000f790 comp rx=0 tx=0).stop 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 -- 172.21.15.45:0/3610704801 shutdown_connections 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639c4670 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 --2- 172.21.15.45:0/3610704801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639b99d0 0x5575639b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 -- 172.21.15.45:0/3610704801 >> 172.21.15.45:0/3610704801 conn(0x5575639b5f60 msgr2=0x5575639b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 -- 172.21.15.45:0/3610704801 shutdown_connections 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 -- 172.21.15.45:0/3610704801 wait complete. 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 -- start start 2024-09-10T10:02:46.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.916+0000 7f6428707740 1 Processor -- start 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5575639c7780 con 0x5575639b99d0 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5575638daf20 con 0x5575639ba2f0 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5575639c7900 con 0x5575639c4bb0 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 10 client.0 _ll_register_callbacks cb 0x5575638a4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33862/0 (socket says 172.21.15.45:33862) 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 -- 172.21.15.45:0/2575047187 learned_addr learned my addr 172.21.15.45:0/2575047187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641ffff640 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 -- 172.21.15.45:0/2575047187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 msgr2=0x5575639c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 -- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 msgr2=0x5575639c17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 -- 172.21.15.45:0/2575047187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575636912b0 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639c4450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6414009230 tx=0x7f6414005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ffbbc645054ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f63ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2575047187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414035040 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2575047187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f64140048c0 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.917+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6410018b20 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2575047187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6414004ac0 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 10 client.14952 fetch_fsmap learned FSMap version 19 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5575639d3010 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2575047187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6410018b20 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2575047187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f64140315b0 con 0x5575639b99d0 2024-09-10T10:02:46.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 10 client.14952 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 10 client.14952 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 20 client.14952 populate_metadata read hostname 'smithi045' 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x5575639d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 4 client.14952 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756392d600 con 0x5575639d5620 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.918+0000 7f641ffff640 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x5575639d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.919+0000 7f641ffff640 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x5575639d6a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b08909f927dc6c3 server_cookie=7a1c5b71ceff8a74 in_seq=0 out_seq=0 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.919+0000 7f63ff7fe640 10 client.14952 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2575047187 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55756392d600 con 0x5575639d5620 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f63ff7fe640 10 client.14952 handle_command_reply: tid=0 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 client.14952 shutdown 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 msgr2=0x5575639c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639c4450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6414009230 tx=0x7f6414005250 comp rx=0 tx=0).stop 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 shutdown_connections 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x5575639d6a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f63ff7fe640 0 client.14952 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 --2- 172.21.15.45:0/2575047187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639c4450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 >> 172.21.15.45:0/2575047187 conn(0x5575639b5f60 msgr2=0x5575639bbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 shutdown_connections 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 1 -- 172.21.15.45:0/2575047187 wait complete. 2024-09-10T10:02:46.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.920+0000 7f6428707740 20 client.14952 trim_cache size 0 max 16384 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 Processor -- start 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 -- start start 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5575639d4dd0 con 0x5575639b99d0 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5575639d54a0 con 0x5575639ba2f0 2024-09-10T10:02:46.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f6428707740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5575639d5b70 con 0x5575639c4bb0 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639d5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639d5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33870/0 (socket says 172.21.15.45:33870) 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 -- 172.21.15.45:0/1909839541 learned_addr learned my addr 172.21.15.45:0/1909839541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641ffff640 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 -- 172.21.15.45:0/1909839541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 msgr2=0x5575639c0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 -- 172.21.15.45:0/1909839541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 msgr2=0x5575639c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 -- 172.21.15.45:0/1909839541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575639b9800 con 0x5575639b99d0 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641ffff640 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.939+0000 7f641f7fe640 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639d5cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f641402a930 tx=0x7f6414032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8597330afcd67bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6405ffb640 1 -- 172.21.15.45:0/1909839541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140046c0 con 0x5575639b99d0 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6405ffb640 1 -- 172.21.15.45:0/1909839541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6414004860 con 0x5575639b99d0 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6405ffb640 1 -- 172.21.15.45:0/1909839541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414003d50 con 0x5575639b99d0 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 -- 172.21.15.45:0/1909839541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 msgr2=0x5575639d5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639d5cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f641402a930 tx=0x7f6414032f90 comp rx=0 tx=0).stop 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 -- 172.21.15.45:0/1909839541 shutdown_connections 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639c4bb0 0x5575639c0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639ba2f0 0x5575639c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 --2- 172.21.15.45:0/1909839541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639b99d0 0x5575639d5cf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 -- 172.21.15.45:0/1909839541 >> 172.21.15.45:0/1909839541 conn(0x5575639b5f60 msgr2=0x5575639c1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 -- 172.21.15.45:0/1909839541 shutdown_connections 2024-09-10T10:02:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.940+0000 7f6428707740 1 -- 172.21.15.45:0/1909839541 wait complete. 2024-09-10T10:02:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 -- start start 2024-09-10T10:02:46.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 Processor -- start 2024-09-10T10:02:46.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639b99d0 0x5575639df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639c4bb0 0x5575639daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5575639d6910 con 0x5575639ba2f0 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5575639d6240 con 0x5575639c4bb0 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5575639d6fe0 con 0x5575639b99d0 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f6428707740 10 client.0 _ll_register_callbacks cb 0x7f6414004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f641ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f641ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639da4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33884/0 (socket says 172.21.15.45:33884) 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.941+0000 7f641ffff640 1 -- 172.21.15.45:0/2972202282 learned_addr learned my addr 172.21.15.45:0/2972202282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639b99d0 0x5575639df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639c4bb0 0x5575639daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641ffff640 1 -- 172.21.15.45:0/2972202282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639b99d0 msgr2=0x5575639df1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641ffff640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639b99d0 0x5575639df1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641ffff640 1 -- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639c4bb0 msgr2=0x5575639daa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641ffff640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639c4bb0 0x5575639daa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641ffff640 1 -- 172.21.15.45:0/2972202282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575639daf60 con 0x5575639ba2f0 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639c4bb0 0x5575639daa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f641ffff640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639da4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f641000ad00 tx=0x7f641000ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3362b1fe5bc4a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f63ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2972202282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641001d040 con 0x5575639ba2f0 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2972202282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f64100187c0 con 0x5575639ba2f0 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6414004860 con 0x5575639ba2f0 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2972202282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6410010660 con 0x5575639ba2f0 2024-09-10T10:02:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f6428707740 10 client.14958 fetch_fsmap learned FSMap version 19 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.942+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557563a3fa40 con 0x5575639ba2f0 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2972202282 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6414004860 con 0x5575639ba2f0 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2972202282 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6410005340 con 0x5575639ba2f0 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f6428707740 10 client.14958 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f6428707740 10 client.14958 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f6428707740 20 client.14958 populate_metadata read hostname 'smithi045' 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f6428707740 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x557563a430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f6428707740 4 client.14958 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "status"}) -- 0x557563927b10 con 0x5575639d5620 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x557563a430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f63ff7fe640 10 client.14958 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:02:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.943+0000 7f641f7fe640 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x557563a430c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=744d0dae8c47ff7b server_cookie=fc07c9c0d681e972 in_seq=0 out_seq=0 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f63ff7fe640 1 -- 172.21.15.45:0/2972202282 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x557563927b10 con 0x5575639d5620 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f63ff7fe640 10 client.14958 handle_command_reply: tid=0 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 client.14958 shutdown 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 msgr2=0x5575639da4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639da4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f641000ad00 tx=0x7f641000ac80 comp rx=0 tx=0).stop 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 shutdown_connections 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5575639d5620 0x557563a430c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5575639c4bb0 0x5575639daa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575639ba2f0 0x5575639da4e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 --2- 172.21.15.45:0/2972202282 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5575639b99d0 0x5575639df1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 >> 172.21.15.45:0/2972202282 conn(0x5575639b5f60 msgr2=0x5575639b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f63ff7fe640 0 client.14958 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:02:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 shutdown_connections 2024-09-10T10:02:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 1 -- 172.21.15.45:0/2972202282 wait complete. 2024-09-10T10:02:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.944+0000 7f6428707740 20 client.14958 trim_cache size 0 max 16384 2024-09-10T10:02:46.953 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T10:02:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "729b10fe-6f5a-11ef-bcea-c7b262605968", 2024-09-10T10:02:46.953 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24488, 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 71.706246088, 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 19, 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 83, 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 82.593467615999998 2024-09-10T10:02:46.954 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64080a2c10 msgr2=0x7f64080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64080a2c10 0x7f64080a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f641809ba20 tx=0x7f6418097630 comp rx=0 tx=0).stop 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420106f10 msgr2=0x7f642019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420106f10 0x7f642019cc20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f641800eaa0 tx=0x7f641800eea0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 shutdown_connections 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f64080a2c10 0x7f64080a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6420106f10 0x7f642019cc20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6420102790 0x7f6420198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 --2- 172.21.15.45:0/8503683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6420101de0 0x7f6420198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 >> 172.21.15.45:0/8503683 conn(0x7f64200fd8c0 msgr2=0x7f64200fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 shutdown_connections 2024-09-10T10:02:46.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:46.945+0000 7f642632e640 1 -- 172.21.15.45:0/8503683 wait complete. 2024-09-10T10:02:47.733 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T10:02:47.733 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T10:02:47.743 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:02:47.743 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T10:02:48.014 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:48.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 -- 172.21.15.45:0/849821795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70102690 msgr2=0x7ffa70108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:48.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/849821795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70102690 0x7ffa70108f20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009ca0 tx=0x7ffa6802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 -- 172.21.15.45:0/849821795 shutdown_connections 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/849821795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70102690 0x7ffa70108f20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/849821795 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa70101cd0 0x7ffa70102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/849821795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa70100ad0 0x7ffa70100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 -- 172.21.15.45:0/849821795 >> 172.21.15.45:0/849821795 conn(0x7ffa700fc280 msgr2=0x7ffa700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 -- 172.21.15.45:0/849821795 shutdown_connections 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.461+0000 7ffa75cc8640 1 -- 172.21.15.45:0/849821795 wait complete. 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 Processor -- start 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 -- start start 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70100ad0 0x7ffa70104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa70101cd0 0x7ffa701030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:48.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa70102690 0x7ffa701035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7010c730 con 0x7ffa70100ad0 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7010c5b0 con 0x7ffa70101cd0 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa75cc8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7010c8b0 con 0x7ffa70102690 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70100ad0 0x7ffa70104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70100ad0 0x7ffa70104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33910/0 (socket says 172.21.15.45:33910) 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/851407026 learned_addr learned my addr 172.21.15.45:0/851407026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa67fff640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa70101cd0 0x7ffa701030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/851407026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa70102690 msgr2=0x7ffa701035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa70102690 0x7ffa701035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/851407026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa70101cd0 msgr2=0x7ffa701030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa70101cd0 0x7ffa701030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 -- 172.21.15.45:0/851407026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa70103eb0 con 0x7ffa70100ad0 2024-09-10T10:02:48.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.462+0000 7ffa6f7fe640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70100ad0 0x7ffa70104a60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e9c0 tx=0x7ffa6000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.463+0000 7ffa6d7fa640 1 -- 172.21.15.45:0/851407026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000cd80 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.463+0000 7ffa6d7fa640 1 -- 172.21.15.45:0/851407026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa600035f0 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.463+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa701a1fd0 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.463+0000 7ffa6d7fa640 1 -- 172.21.15.45:0/851407026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa60010690 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.463+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a24d0 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.464+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa701a2760 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.466+0000 7ffa6d7fa640 1 -- 172.21.15.45:0/851407026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffa600037a0 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.466+0000 7ffa6d7fa640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffa5c0a2c90 0x7ffa5c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.466+0000 7ffa67fff640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffa5c0a2c90 0x7ffa5c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.466+0000 7ffa6d7fa640 1 -- 172.21.15.45:0/851407026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ffa600cf690 con 0x7ffa70100ad0 2024-09-10T10:02:48.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.467+0000 7ffa67fff640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffa5c0a2c90 0x7ffa5c0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa701047e0 tx=0x7ffa5800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.474+0000 7ffa6d7fa640 1 -- 172.21.15.45:0/851407026 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa6009c4a0 con 0x7ffa70100ad0 2024-09-10T10:02:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:48 smithi045 ceph-mon[31114]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:48.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.672+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ffa701a2930 con 0x7ffa5c0a2c90 2024-09-10T10:02:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.673+0000 7ffa6d7fa640 1 -- 172.21.15.45:0/851407026 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ffa701a2930 con 0x7ffa5c0a2c90 2024-09-10T10:02:48.675 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-09-10T10:02:48.675 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-09-10T10:02:48.675 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-09-10T10:02:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffa5c0a2c90 msgr2=0x7ffa5c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffa5c0a2c90 0x7ffa5c0a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffa701047e0 tx=0x7ffa5800a4c0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70100ad0 msgr2=0x7ffa70104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70100ad0 0x7ffa70104a60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e9c0 tx=0x7ffa6000ee80 comp rx=0 tx=0).stop 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 shutdown_connections 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7ffa5c0a2c90 0x7ffa5c0a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa70102690 0x7ffa701035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa70101cd0 0x7ffa701030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 --2- 172.21.15.45:0/851407026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa70100ad0 0x7ffa70104a60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 >> 172.21.15.45:0/851407026 conn(0x7ffa700fc280 msgr2=0x7ffa700fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 shutdown_connections 2024-09-10T10:02:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:48.675+0000 7ffa75cc8640 1 -- 172.21.15.45:0/851407026 wait complete. 2024-09-10T10:02:48.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:48 smithi162 ceph-mon[34963]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:48.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:48 smithi047 ceph-mon[35244]: pgmap v312: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:49.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T10:02:49.603 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:50.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 -- 172.21.15.45:0/3319291134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c101d10 msgr2=0x7f1a6c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 --2- 172.21.15.45:0/3319291134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c101d10 0x7f1a6c102190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a6002a410 comp rx=0 tx=0).stop 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 -- 172.21.15.45:0/3319291134 shutdown_connections 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 --2- 172.21.15.45:0/3319291134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6c1026d0 0x7f1a6c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 --2- 172.21.15.45:0/3319291134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c101d10 0x7f1a6c102190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 --2- 172.21.15.45:0/3319291134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a6c100b10 0x7f1a6c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 -- 172.21.15.45:0/3319291134 >> 172.21.15.45:0/3319291134 conn(0x7f1a6c0fc2a0 msgr2=0x7f1a6c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 -- 172.21.15.45:0/3319291134 shutdown_connections 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 -- 172.21.15.45:0/3319291134 wait complete. 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 Processor -- start 2024-09-10T10:02:50.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.100+0000 7f1a72626640 1 -- start start 2024-09-10T10:02:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a72626640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c100b10 0x7f1a6c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a72626640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6c101d10 0x7f1a6c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a72626640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a6c1026d0 0x7f1a6c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a72626640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c10c7f0 con 0x7f1a6c100b10 2024-09-10T10:02:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a72626640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c10c670 con 0x7f1a6c1026d0 2024-09-10T10:02:50.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a72626640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c10c970 con 0x7f1a6c101d10 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c100b10 0x7f1a6c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c100b10 0x7f1a6c06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33936/0 (socket says 172.21.15.45:33936) 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 -- 172.21.15.45:0/449111834 learned_addr learned my addr 172.21.15.45:0/449111834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a70bc3640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a6c1026d0 0x7f1a6c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6b7fe640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6c101d10 0x7f1a6c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 -- 172.21.15.45:0/449111834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6c101d10 msgr2=0x7f1a6c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6c101d10 0x7f1a6c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 -- 172.21.15.45:0/449111834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a6c1026d0 msgr2=0x7f1a6c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a6c1026d0 0x7f1a6c19f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6bfff640 1 -- 172.21.15.45:0/449111834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c19fc00 con 0x7f1a6c100b10 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.101+0000 7f1a6b7fe640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6c101d10 0x7f1a6c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.102+0000 7f1a6bfff640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c100b10 0x7f1a6c06b2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00d770 tx=0x7f1a5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.102+0000 7f1a697fa640 1 -- 172.21.15.45:0/449111834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c017070 con 0x7f1a6c100b10 2024-09-10T10:02:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.102+0000 7f1a697fa640 1 -- 172.21.15.45:0/449111834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a5c009c30 con 0x7f1a6c100b10 2024-09-10T10:02:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.102+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c19fe90 con 0x7f1a6c100b10 2024-09-10T10:02:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.102+0000 7f1a697fa640 1 -- 172.21.15.45:0/449111834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c003dd0 con 0x7f1a6c100b10 2024-09-10T10:02:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.102+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c1a03c0 con 0x7f1a6c100b10 2024-09-10T10:02:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.103+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6c100f10 con 0x7f1a6c100b10 2024-09-10T10:02:50.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.105+0000 7f1a697fa640 1 -- 172.21.15.45:0/449111834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1a5c003f70 con 0x7f1a6c100b10 2024-09-10T10:02:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.105+0000 7f1a697fa640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1a540a2c10 0x7f1a540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.106+0000 7f1a6b7fe640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1a540a2c10 0x7f1a540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.106+0000 7f1a697fa640 1 -- 172.21.15.45:0/449111834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1a5c0cefb0 con 0x7f1a6c100b10 2024-09-10T10:02:50.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.106+0000 7f1a6b7fe640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1a540a2c10 0x7f1a540a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a60000c00 tx=0x7f1a60031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:50.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.109+0000 7f1a697fa640 1 -- 172.21.15.45:0/449111834 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a5c009de0 con 0x7f1a6c100b10 2024-09-10T10:02:50.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.291+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1a6c062630 con 0x7f1a540a2c10 2024-09-10T10:02:50.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.303+0000 7f1a697fa640 1 -- 172.21.15.45:0/449111834 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f1a6c062630 con 0x7f1a540a2c10 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (88s) 13s ago 88s 16.6M - 19.1.1-305-g1cf819b7 41d671128357 ff9749f130a3 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi047 running (86s) 18s ago 85s 28.5M - 19.1.1-305-g1cf819b7 41d671128357 f6d7311f537f 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi162 running (99s) 18s ago 99s 14.3M - 19.1.1-305-g1cf819b7 41d671128357 b1281eee2588 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (107s) 13s ago 106s 16.5M - 19.1.1-305-g1cf819b7 41d671128357 acb839177401 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi047 running (104s) 18s ago 103s 13.4M - 19.1.1-305-g1cf819b7 41d671128357 0ca7f833c8d3 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi162 running (101s) 18s ago 101s 13.5M - 19.1.1-305-g1cf819b7 41d671128357 55f6f8d1ef32 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (92s) 13s ago 90s 13.2M - 19.1.1-305-g1cf819b7 41d671128357 f0e788d91c92 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi047 running (97s) 18s ago 96s 14.3M - 19.1.1-305-g1cf819b7 41d671128357 5548595c5848 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi162 running (94s) 18s ago 94s 18.0M - 19.1.1-305-g1cf819b7 41d671128357 8648587df709 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (110s) 13s ago 109s 16.8M - 19.1.1-305-g1cf819b7 41d671128357 bf77fe694512 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi047 running (113s) 18s ago 112s 16.6M - 19.1.1-305-g1cf819b7 41d671128357 1b558e67b92b 2024-09-10T10:02:50.306 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi162 running (115s) 18s ago 115s 13.5M - 19.1.1-305-g1cf819b7 41d671128357 5564991b156e 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (8m) 13s ago 8m 508M - 19.1.1-305-g1cf819b7 41d671128357 5586f3d8900f 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi047 *:8443,8765 running (7m) 18s ago 7m 440M - 19.1.1-305-g1cf819b7 41d671128357 dd075593474f 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi162 *:8443,8765 running (7m) 18s ago 7m 443M - 19.1.1-305-g1cf819b7 41d671128357 94a2f2b5237c 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (8m) 13s ago 8m 54.0M 2048M 19.1.1-305-g1cf819b7 41d671128357 ab3b8fe2420a 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi047 running (7m) 18s ago 7m 41.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 267f885128f0 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi162 running (7m) 18s ago 7m 41.3M 2048M 19.1.1-305-g1cf819b7 41d671128357 ca3816f4365a 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (6m) 13s ago 6m 49.1M 4273M 19.1.1-305-g1cf819b7 41d671128357 5e35b0a99267 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi047 running (6m) 18s ago 6m 50.4M 4273M 19.1.1-305-g1cf819b7 41d671128357 02a376af6bac 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi162 running (5m) 18s ago 5m 49.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 9f3fc640bde9 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (5m) 13s ago 5m 46.8M 4273M 19.1.1-305-g1cf819b7 41d671128357 b11061e943cf 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi047 running (5m) 18s ago 5m 47.5M 4273M 19.1.1-305-g1cf819b7 41d671128357 823c044f6b9f 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi162 running (4m) 18s ago 4m 55.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 4c258e15f632 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (4m) 13s ago 4m 42.8M 4273M 19.1.1-305-g1cf819b7 41d671128357 8bcbcd543097 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi047 running (3m) 18s ago 3m 54.2M 4273M 19.1.1-305-g1cf819b7 41d671128357 152744f346c4 2024-09-10T10:02:50.307 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi162 running (3m) 18s ago 3m 48.3M 4272M 19.1.1-305-g1cf819b7 41d671128357 2a159fc646f5 2024-09-10T10:02:50.308 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (3m) 13s ago 3m 43.6M 4273M 19.1.1-305-g1cf819b7 41d671128357 75985b06322b 2024-09-10T10:02:50.308 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi047 running (2m) 18s ago 2m 46.9M 4273M 19.1.1-305-g1cf819b7 41d671128357 143d2c57d1b2 2024-09-10T10:02:50.308 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi162 running (2m) 18s ago 2m 51.9M 4272M 19.1.1-305-g1cf819b7 41d671128357 69daefcab44f 2024-09-10T10:02:50.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1a540a2c10 msgr2=0x7f1a540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1a540a2c10 0x7f1a540a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a60000c00 tx=0x7f1a60031400 comp rx=0 tx=0).stop 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c100b10 msgr2=0x7f1a6c06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c100b10 0x7f1a6c06b2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00d770 tx=0x7f1a5c00dc30 comp rx=0 tx=0).stop 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 shutdown_connections 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f1a540a2c10 0x7f1a540a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a6c1026d0 0x7f1a6c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1a6c101d10 0x7f1a6c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 --2- 172.21.15.45:0/449111834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a6c100b10 0x7f1a6c06b2f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.305+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 >> 172.21.15.45:0/449111834 conn(0x7f1a6c0fc2a0 msgr2=0x7f1a6c105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.306+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 shutdown_connections 2024-09-10T10:02:50.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:50.306+0000 7f1a72626640 1 -- 172.21.15.45:0/449111834 wait complete. 2024-09-10T10:02:50.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:50 smithi045 ceph-mon[31114]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:50.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:50 smithi045 ceph-mon[31114]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:50 smithi162 ceph-mon[34963]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:50.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:50 smithi162 ceph-mon[34963]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:50.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:50 smithi047 ceph-mon[35244]: pgmap v313: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:50.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:50 smithi047 ceph-mon[35244]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:50.980 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T10:02:51.238 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:51.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:51 smithi045 ceph-mon[31114]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:51.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 -- 172.21.15.45:0/1830834485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 msgr2=0x7fd790107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 --2- 172.21.15.45:0/1830834485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790107380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd780009900 tx=0x7fd78002a410 comp rx=0 tx=0).stop 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 -- 172.21.15.45:0/1830834485 shutdown_connections 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 --2- 172.21.15.45:0/1830834485 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd7901078c0 0x7fd790109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 --2- 172.21.15.45:0/1830834485 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790107380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 --2- 172.21.15.45:0/1830834485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd790102660 0x7fd790104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 -- 172.21.15.45:0/1830834485 >> 172.21.15.45:0/1830834485 conn(0x7fd7900fc280 msgr2=0x7fd7900fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 -- 172.21.15.45:0/1830834485 shutdown_connections 2024-09-10T10:02:51.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 -- 172.21.15.45:0/1830834485 wait complete. 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.681+0000 7fd7980a9640 1 Processor -- start 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd7980a9640 1 -- start start 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd7980a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd790102660 0x7fd790197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd7980a9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd7980a9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7901078c0 0x7fd79019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd7980a9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd79010d860 con 0x7fd790104f90 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd7980a9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd79010d6e0 con 0x7fd7901078c0 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd7980a9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd79010d9e0 con 0x7fd790102660 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33954/0 (socket says 172.21.15.45:33954) 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 -- 172.21.15.45:0/2986228053 learned_addr learned my addr 172.21.15.45:0/2986228053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd796646640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7901078c0 0x7fd79019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795e45640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd790102660 0x7fd790197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 -- 172.21.15.45:0/2986228053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd790102660 msgr2=0x7fd790197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd790102660 0x7fd790197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 -- 172.21.15.45:0/2986228053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7901078c0 msgr2=0x7fd79019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7901078c0 0x7fd79019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd795644640 1 -- 172.21.15.45:0/2986228053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79019fb40 con 0x7fd790104f90 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.682+0000 7fd796646640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7901078c0 0x7fd79019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:51.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.683+0000 7fd795644640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790198450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd780031610 tx=0x7fd7800043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.683+0000 7fd786ffd640 1 -- 172.21.15.45:0/2986228053 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78003f030 con 0x7fd790104f90 2024-09-10T10:02:51.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.683+0000 7fd786ffd640 1 -- 172.21.15.45:0/2986228053 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd780003ce0 con 0x7fd790104f90 2024-09-10T10:02:51.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.683+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79019fdd0 con 0x7fd790104f90 2024-09-10T10:02:51.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.683+0000 7fd786ffd640 1 -- 172.21.15.45:0/2986228053 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780035070 con 0x7fd790104f90 2024-09-10T10:02:51.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.683+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7901a0270 con 0x7fd790104f90 2024-09-10T10:02:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.684+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7900fdd70 con 0x7fd790104f90 2024-09-10T10:02:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.685+0000 7fd786ffd640 1 -- 172.21.15.45:0/2986228053 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd780042030 con 0x7fd790104f90 2024-09-10T10:02:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.686+0000 7fd786ffd640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd77c0a2cc0 0x7fd77c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.686+0000 7fd795e45640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd77c0a2cc0 0x7fd77c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.686+0000 7fd786ffd640 1 -- 172.21.15.45:0/2986228053 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd7800ef650 con 0x7fd790104f90 2024-09-10T10:02:51.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.686+0000 7fd795e45640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd77c0a2cc0 0x7fd77c0a5180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bbe0 tx=0x7fd78c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:51.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.690+0000 7fd786ffd640 1 -- 172.21.15.45:0/2986228053 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7800bbe10 con 0x7fd790104f90 2024-09-10T10:02:51.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.873+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd790062630 con 0x7fd77c0a2cc0 2024-09-10T10:02:51.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.879+0000 7fd786ffd640 1 -- 172.21.15.45:0/2986228053 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+985 (secure 0 0 0) 0x7fd790062630 con 0x7fd77c0a2cc0 2024-09-10T10:02:51.881 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T10:02:51.881 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 20s ago 119s smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi047=b;smithi047=e;smithi047=h;smithi047=k;smithi162=c;smithi162=f;smithi162=i;smithi162=l;count:12 2024-09-10T10:02:51.882 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 20s ago 7m smithi045=x;smithi047=y;smithi162=z;count:3 2024-09-10T10:02:51.882 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 20s ago 7m smithi045:172.21.15.45=a;smithi047:172.21.15.47=b;smithi162:172.21.15.162=c;count:3 2024-09-10T10:02:51.882 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 20s ago - 2024-09-10T10:02:51.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd77c0a2cc0 msgr2=0x7fd77c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd77c0a2cc0 0x7fd77c0a5180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00bbe0 tx=0x7fd78c00b390 comp rx=0 tx=0).stop 2024-09-10T10:02:51.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 msgr2=0x7fd790198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:51.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790198450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd780031610 tx=0x7fd7800043b0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 shutdown_connections 2024-09-10T10:02:51.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fd77c0a2cc0 0x7fd77c0a5180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7901078c0 0x7fd79019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd790104f90 0x7fd790198450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 --2- 172.21.15.45:0/2986228053 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd790102660 0x7fd790197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:51.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 >> 172.21.15.45:0/2986228053 conn(0x7fd7900fc280 msgr2=0x7fd790103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:51.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 shutdown_connections 2024-09-10T10:02:51.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:51.881+0000 7fd7980a9640 1 -- 172.21.15.45:0/2986228053 wait complete. 2024-09-10T10:02:51.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:51 smithi162 ceph-mon[34963]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:51.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:51 smithi047 ceph-mon[35244]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:52.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T10:02:52.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:52 smithi045 ceph-mon[31114]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:52.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:52 smithi162 ceph-mon[34963]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:52.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:52 smithi047 ceph-mon[35244]: pgmap v314: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 -- 172.21.15.45:0/4264681061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341037c0 msgr2=0x7fef3410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 --2- 172.21.15.45:0/4264681061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341037c0 0x7fef3410a050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009ca0 tx=0x7fef2c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 -- 172.21.15.45:0/4264681061 shutdown_connections 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 --2- 172.21.15.45:0/4264681061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef341037c0 0x7fef3410a050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 --2- 172.21.15.45:0/4264681061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef34102e00 0x7fef34103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 --2- 172.21.15.45:0/4264681061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef34101c00 0x7fef34102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 -- 172.21.15.45:0/4264681061 >> 172.21.15.45:0/4264681061 conn(0x7fef340fd390 msgr2=0x7fef340ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 -- 172.21.15.45:0/4264681061 shutdown_connections 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.231+0000 7fef3af91640 1 -- 172.21.15.45:0/4264681061 wait complete. 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 Processor -- start 2024-09-10T10:02:53.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 -- start start 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef34101c00 0x7fef34075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef34102e00 0x7fef340745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef341037c0 0x7fef34074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef3410d900 con 0x7fef34101c00 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef3410d780 con 0x7fef341037c0 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3af91640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fef3410da80 con 0x7fef34102e00 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef38d2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef34101c00 0x7fef34075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef38d2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef34101c00 0x7fef34075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44014/0 (socket says 172.21.15.45:44014) 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef38d2d640 1 -- 172.21.15.45:0/3912460355 learned_addr learned my addr 172.21.15.45:0/3912460355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef33fff640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef34102e00 0x7fef340745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.232+0000 7fef3952e640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef341037c0 0x7fef34074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef38d2d640 1 -- 172.21.15.45:0/3912460355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef34102e00 msgr2=0x7fef340745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef38d2d640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef34102e00 0x7fef340745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef38d2d640 1 -- 172.21.15.45:0/3912460355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef341037c0 msgr2=0x7fef34074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef38d2d640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef341037c0 0x7fef34074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef38d2d640 1 -- 172.21.15.45:0/3912460355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef340753e0 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef38d2d640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef34101c00 0x7fef34075f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fef2800b450 tx=0x7fef2800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef31ffb640 1 -- 172.21.15.45:0/3912460355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28015020 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef31ffb640 1 -- 172.21.15.45:0/3912460355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef28011d70 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef341a1fa0 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef31ffb640 1 -- 172.21.15.45:0/3912460355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2800f340 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.233+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef341a2480 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.234+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef34108f30 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.235+0000 7fef31ffb640 1 -- 172.21.15.45:0/3912460355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fef28018020 con 0x7fef34101c00 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.236+0000 7fef31ffb640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fef240a2c10 0x7fef240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:53.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.236+0000 7fef33fff640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fef240a2c10 0x7fef240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:53.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.236+0000 7fef31ffb640 1 -- 172.21.15.45:0/3912460355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fef280cec20 con 0x7fef34101c00 2024-09-10T10:02:53.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.236+0000 7fef33fff640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fef240a2c10 0x7fef240a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef20005e10 tx=0x7fef20005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:53.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.240+0000 7fef31ffb640 1 -- 172.21.15.45:0/3912460355 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef2801f020 con 0x7fef34101c00 2024-09-10T10:02:53.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.425+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fef34062630 con 0x7fef240a2c10 2024-09-10T10:02:53.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.426+0000 7fef31ffb640 1 -- 172.21.15.45:0/3912460355 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fef34062630 con 0x7fef240a2c10 2024-09-10T10:02:53.428 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-09-10T10:02:53.428 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-09-10T10:02:53.428 INFO:teuthology.orchestra.run.smithi045.stdout:smithi047 172.21.15.47 2024-09-10T10:02:53.429 INFO:teuthology.orchestra.run.smithi045.stdout:smithi162 172.21.15.162 2024-09-10T10:02:53.429 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fef240a2c10 msgr2=0x7fef240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fef240a2c10 0x7fef240a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef20005e10 tx=0x7fef20005da0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef34101c00 msgr2=0x7fef34075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef34101c00 0x7fef34075f90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fef2800b450 tx=0x7fef2800b910 comp rx=0 tx=0).stop 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 shutdown_connections 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fef240a2c10 0x7fef240a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef341037c0 0x7fef34074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef34102e00 0x7fef340745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 --2- 172.21.15.45:0/3912460355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef34101c00 0x7fef34075f90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.428+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 >> 172.21.15.45:0/3912460355 conn(0x7fef340fd390 msgr2=0x7fef34106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:53.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.429+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 shutdown_connections 2024-09-10T10:02:53.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:53.429+0000 7fef3af91640 1 -- 172.21.15.45:0/3912460355 wait complete. 2024-09-10T10:02:53.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:53 smithi045 ceph-mon[31114]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:53 smithi162 ceph-mon[34963]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:53.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:53 smithi047 ceph-mon[35244]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:54.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T10:02:54.270 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:54.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:54 smithi045 ceph-mon[31114]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:54.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:54 smithi045 ceph-mon[31114]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:54.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.765+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3868694554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4100ad0 msgr2=0x7f8ed4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.765+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3868694554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4100ad0 0x7f8ed4100ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009900 tx=0x7f8ec402a410 comp rx=0 tx=0).stop 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3868694554 shutdown_connections 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3868694554 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed4102690 0x7f8ed4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3868694554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ed4101cd0 0x7f8ed4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3868694554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4100ad0 0x7f8ed4100ed0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3868694554 >> 172.21.15.45:0/3868694554 conn(0x7f8ed40fc280 msgr2=0x7f8ed40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3868694554 shutdown_connections 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3868694554 wait complete. 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 Processor -- start 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.766+0000 7f8eda1b6640 1 -- start start 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ed4100ad0 0x7f8ed4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4101cd0 0x7f8ed41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed4102690 0x7f8ed419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410c760 con 0x7f8ed4101cd0 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410c5e0 con 0x7f8ed4100ad0 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed410c8e0 con 0x7f8ed4102690 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4101cd0 0x7f8ed41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4101cd0 0x7f8ed41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44040/0 (socket says 172.21.15.45:44040) 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 -- 172.21.15.45:0/3565044306 learned_addr learned my addr 172.21.15.45:0/3565044306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ed37fe640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ed4100ad0 0x7f8ed4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ed3fff640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed4102690 0x7f8ed419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 -- 172.21.15.45:0/3565044306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed4102690 msgr2=0x7f8ed419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed4102690 0x7f8ed419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 -- 172.21.15.45:0/3565044306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ed4100ad0 msgr2=0x7f8ed4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ed4100ad0 0x7f8ed4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 -- 172.21.15.45:0/3565044306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed419fab0 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ed3fff640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed4102690 0x7f8ed419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ecaffd640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4101cd0 0x7f8ed41983c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c920 tx=0x7f8ebc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ed17fa640 1 -- 172.21.15.45:0/3565044306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc007c70 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ed17fa640 1 -- 172.21.15.45:0/3565044306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ebc007e10 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed419fda0 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8ed17fa640 1 -- 172.21.15.45:0/3565044306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc018890 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.767+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a0260 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.769+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed41a0520 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.770+0000 7f8ed17fa640 1 -- 172.21.15.45:0/3565044306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8ebc018a30 con 0x7f8ed4101cd0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.771+0000 7f8ed17fa640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f8ec00a2cc0 0x7f8ec00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.771+0000 7f8ed37fe640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f8ec00a2cc0 0x7f8ec00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:54.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.771+0000 7f8ed17fa640 1 -- 172.21.15.45:0/3565044306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8ebc0cf280 con 0x7f8ed4101cd0 2024-09-10T10:02:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.771+0000 7f8ed37fe640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f8ec00a2cc0 0x7f8ec00a5180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005ab0 tx=0x7f8ec4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:54.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.775+0000 7f8ed17fa640 1 -- 172.21.15.45:0/3565044306 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ebc014030 con 0x7f8ed4101cd0 2024-09-10T10:02:54.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:54 smithi162 ceph-mon[34963]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:54.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:54 smithi162 ceph-mon[34963]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:54.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:54 smithi047 ceph-mon[35244]: pgmap v315: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:02:54.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:54 smithi047 ceph-mon[35244]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:54.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.957+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 --> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8ed41a06f0 con 0x7f8ec00a2cc0 2024-09-10T10:02:54.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.959+0000 7f8ed17fa640 1 -- 172.21.15.45:0/3565044306 <== mgr.14150 v2:172.21.15.45:6800/952575454 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8ed41a06f0 con 0x7f8ec00a2cc0 2024-09-10T10:02:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T10:02:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-10T10:02:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-10T10:02:54.961 INFO:teuthology.orchestra.run.smithi045.stdout:smithi162 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JU400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T10:02:54.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f8ec00a2cc0 msgr2=0x7f8ec00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f8ec00a2cc0 0x7f8ec00a5180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005ab0 tx=0x7f8ec4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4101cd0 msgr2=0x7f8ed41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4101cd0 0x7f8ed41983c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c920 tx=0x7f8ebc00cde0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 shutdown_connections 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f8ec00a2cc0 0x7f8ec00a5180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ed4102690 0x7f8ed419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ed4101cd0 0x7f8ed41983c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 --2- 172.21.15.45:0/3565044306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ed4100ad0 0x7f8ed4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 >> 172.21.15.45:0/3565044306 conn(0x7f8ed40fc280 msgr2=0x7f8ed40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 shutdown_connections 2024-09-10T10:02:54.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:54.961+0000 7f8eda1b6640 1 -- 172.21.15.45:0/3565044306 wait complete. 2024-09-10T10:02:55.613 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T10:02:55.622 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:02:55.622 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T10:02:55.885 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.345+0000 7f9459c43640 1 -- 172.21.15.45:0/3848362213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9454104fd0 msgr2=0x7f94541073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.345+0000 7f9459c43640 1 --2- 172.21.15.45:0/3848362213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9454104fd0 0x7f94541073c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f943c009900 tx=0x7f943c02a410 comp rx=0 tx=0).stop 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 -- 172.21.15.45:0/3848362213 shutdown_connections 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 --2- 172.21.15.45:0/3848362213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9454107900 0x7f9454109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 --2- 172.21.15.45:0/3848362213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9454104fd0 0x7f94541073c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 --2- 172.21.15.45:0/3848362213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94541026a0 0x7f9454104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 -- 172.21.15.45:0/3848362213 >> 172.21.15.45:0/3848362213 conn(0x7f94540fc2a0 msgr2=0x7f94540fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 -- 172.21.15.45:0/3848362213 shutdown_connections 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 -- 172.21.15.45:0/3848362213 wait complete. 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 Processor -- start 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.346+0000 7f9459c43640 1 -- start start 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94541026a0 0x7f9454197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9454104fd0 0x7f9454198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9454107900 0x7f945419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f945410d8a0 con 0x7f94541026a0 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f945410d720 con 0x7f9454107900 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f945410da20 con 0x7f9454104fd0 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94541026a0 0x7f9454197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94541026a0 0x7f9454197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44064/0 (socket says 172.21.15.45:44064) 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 -- 172.21.15.45:0/2788682708 learned_addr learned my addr 172.21.15.45:0/2788682708 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f944affd640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9454104fd0 0x7f9454198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9453fff640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9454107900 0x7f945419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 -- 172.21.15.45:0/2788682708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9454104fd0 msgr2=0x7f9454198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9454104fd0 0x7f9454198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 -- 172.21.15.45:0/2788682708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9454107900 msgr2=0x7f945419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9454107900 0x7f945419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 -- 172.21.15.45:0/2788682708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945419faf0 con 0x7f94541026a0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9453fff640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9454107900 0x7f945419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94537fe640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94541026a0 0x7f9454197ec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f944400c920 tx=0x7f944400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94517fa640 1 -- 172.21.15.45:0/2788682708 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444007c70 con 0x7f94541026a0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945419fde0 con 0x7f94541026a0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94517fa640 1 -- 172.21.15.45:0/2788682708 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9444007e10 con 0x7f94541026a0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f94517fa640 1 -- 172.21.15.45:0/2788682708 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444018890 con 0x7f94541026a0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.347+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94541a0240 con 0x7f94541026a0 2024-09-10T10:02:56.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.348+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94540fdd90 con 0x7f94541026a0 2024-09-10T10:02:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.350+0000 7f94517fa640 1 -- 172.21.15.45:0/2788682708 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9444018a30 con 0x7f94541026a0 2024-09-10T10:02:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.350+0000 7f94517fa640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94400a2c40 0x7f94400a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.351+0000 7f94517fa640 1 -- 172.21.15.45:0/2788682708 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f94440cf5f0 con 0x7f94541026a0 2024-09-10T10:02:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.351+0000 7f944affd640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94400a2c40 0x7f94400a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.351+0000 7f944affd640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94400a2c40 0x7f94400a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f943c006010 tx=0x7f943c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:56.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.354+0000 7f94517fa640 1 -- 172.21.15.45:0/2788682708 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9444014030 con 0x7f94541026a0 2024-09-10T10:02:56.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.565+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f945402d7c0 con 0x7f94541026a0 2024-09-10T10:02:56.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:56 smithi045 ceph-mon[31114]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:56.567 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:56 smithi045 ceph-mon[31114]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.565+0000 7f94517fa640 1 -- 172.21.15.45:0/2788682708 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+3042 (secure 0 0 0) 0x7f944400de20 con 0x7f94541026a0 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stdout:e19 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stdout:btime 2024-09-10T10:01:44:271138+0000 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:56.568 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 19 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-10T10:01:34.924889+0000 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-09-10T10:01:43.932833+0000 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2024-09-10T10:02:56.569 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 1 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:in 0 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24488} 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3] 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:qdb_cluster leader: 24488 members: 24488 2024-09-10T10:02:56.570 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24488} state up:active seq 4 addr [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.162:6838/2065513838,v1:172.21.15.162:6839/2065513838] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.45:6834/2062348864,v1:172.21.15.45:6835/2062348864] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.45:6836/1314634033,v1:172.21.15.45:6837/1314634033] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{-1:14592} state up:standby seq 1 addr [v2:172.21.15.45:6838/743802171,v1:172.21.15.45:6839/743802171] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{-1:14598} state up:standby seq 1 addr [v2:172.21.15.45:6840/511130082,v1:172.21.15.45:6841/511130082] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{-1:24415} state up:standby seq 1 addr [v2:172.21.15.162:6832/531637764,v1:172.21.15.162:6833/531637764] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{-1:24416} state up:standby seq 1 addr [v2:172.21.15.47:6832/2229149639,v1:172.21.15.47:6833/2229149639] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{-1:24437} state up:standby seq 1 addr [v2:172.21.15.47:6834/2176252394,v1:172.21.15.47:6835/2176252394] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{-1:24445} state up:standby seq 1 addr [v2:172.21.15.162:6834/1763949384,v1:172.21.15.162:6835/1763949384] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.571 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{-1:24457} state up:standby seq 1 addr [v2:172.21.15.162:6836/3408858695,v1:172.21.15.162:6837/3408858695] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.572 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{-1:24458} state up:standby seq 1 addr [v2:172.21.15.47:6836/2773219506,v1:172.21.15.47:6837/2773219506] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T10:02:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-09-10T10:02:56.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.567+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94400a2c40 msgr2=0x7f94400a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.567+0000 7f9459c43640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94400a2c40 0x7f94400a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f943c006010 tx=0x7f943c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.567+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94541026a0 msgr2=0x7f9454197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.567+0000 7f9459c43640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94541026a0 0x7f9454197ec0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f944400c920 tx=0x7f944400cde0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 shutdown_connections 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f94400a2c40 0x7f94400a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9454107900 0x7f945419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9454104fd0 0x7f9454198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 --2- 172.21.15.45:0/2788682708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94541026a0 0x7f9454197ec0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 >> 172.21.15.45:0/2788682708 conn(0x7f94540fc2a0 msgr2=0x7f94541030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 shutdown_connections 2024-09-10T10:02:56.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:56.568+0000 7f9459c43640 1 -- 172.21.15.45:0/2788682708 wait complete. 2024-09-10T10:02:56.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:56 smithi162 ceph-mon[34963]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:56.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:56 smithi162 ceph-mon[34963]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:56.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:56 smithi047 ceph-mon[35244]: pgmap v316: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:56.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:56 smithi047 ceph-mon[35244]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:02:57.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T10:02:57.472 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:57.747 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:57 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2788682708' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:02:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:57 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2788682708' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:02:57.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:57 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2788682708' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 -- 172.21.15.45:0/602907725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 msgr2=0x7f574c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 --2- 172.21.15.45:0/602907725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c102190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f573c009900 tx=0x7f573c02a410 comp rx=0 tx=0).stop 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 -- 172.21.15.45:0/602907725 shutdown_connections 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 --2- 172.21.15.45:0/602907725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c1026d0 0x7f574c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 --2- 172.21.15.45:0/602907725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c102190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 --2- 172.21.15.45:0/602907725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c100b10 0x7f574c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 -- 172.21.15.45:0/602907725 >> 172.21.15.45:0/602907725 conn(0x7f574c0fc2a0 msgr2=0x7f574c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 -- 172.21.15.45:0/602907725 shutdown_connections 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.944+0000 7f57535ea640 1 -- 172.21.15.45:0/602907725 wait complete. 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 Processor -- start 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 -- start start 2024-09-10T10:02:57.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c100b10 0x7f574c06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c1026d0 0x7f574c19f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f574c10c770 con 0x7f574c101d10 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f574c10c5f0 con 0x7f574c100b10 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f57535ea640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f574c10c8f0 con 0x7f574c1026d0 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f5750b85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f5750b85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44090/0 (socket says 172.21.15.45:44090) 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f5750b85640 1 -- 172.21.15.45:0/3153407213 learned_addr learned my addr 172.21.15.45:0/3153407213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.945+0000 7f5751386640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c100b10 0x7f574c06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f5751b87640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c1026d0 0x7f574c19f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f5750b85640 1 -- 172.21.15.45:0/3153407213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c1026d0 msgr2=0x7f574c19f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f5750b85640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c1026d0 0x7f574c19f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f5750b85640 1 -- 172.21.15.45:0/3153407213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c100b10 msgr2=0x7f574c06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f5750b85640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c100b10 0x7f574c06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f5750b85640 1 -- 172.21.15.45:0/3153407213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c19fb30 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f5750b85640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c19cc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f573c009230 tx=0x7f573c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f57427fc640 1 -- 172.21.15.45:0/3153407213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c03f030 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c19fd60 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f57427fc640 1 -- 172.21.15.45:0/3153407213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f573c003ce0 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f57427fc640 1 -- 172.21.15.45:0/3153407213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c035070 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.946+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c1a0230 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.947+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c107f10 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.949+0000 7f57427fc640 1 -- 172.21.15.45:0/3153407213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f573c042030 con 0x7f574c101d10 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.949+0000 7f57427fc640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f57380a2c10 0x7f57380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.950+0000 7f5751386640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f57380a2c10 0x7f57380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:57.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.950+0000 7f57427fc640 1 -- 172.21.15.45:0/3153407213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f573c0efda0 con 0x7f574c101d10 2024-09-10T10:02:57.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.950+0000 7f5751386640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f57380a2c10 0x7f57380a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f574c101b70 tx=0x7f574800a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:57.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:57.953+0000 7f57427fc640 1 -- 172.21.15.45:0/3153407213 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f573c0bcb50 con 0x7f574c101d10 2024-09-10T10:02:58.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.128+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f574c02d9c0 con 0x7f574c101d10 2024-09-10T10:02:58.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.129+0000 7f57427fc640 1 -- 172.21.15.45:0/3153407213 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5266 (secure 0 0 0) 0x7f573c0bc450 con 0x7f574c101d10 2024-09-10T10:02:58.132 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 83 2024-09-10T10:02:58.132 INFO:teuthology.orchestra.run.smithi045.stdout:fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T10:02:58.132 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-09-10T09:53:53.416864+0000 2024-09-10T10:02:58.132 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-09-10T10:01:50.297428+0000 2024-09-10T10:02:58.132 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T10:02:58.132 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 26 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2024-09-10T10:02:58.133 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/378340133,v1:172.21.15.45:6803/378340133] [v2:172.21.15.45:6804/378340133,v1:172.21.15.45:6805/378340133] exists,up edb5ca81-e36d-4acd-9f67-35e5fb644772 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6800/3842809154,v1:172.21.15.47:6801/3842809154] [v2:172.21.15.47:6802/3842809154,v1:172.21.15.47:6803/3842809154] exists,up 20cd1eda-0149-4eab-a5ff-9463c2b556fa 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6800/572103448,v1:172.21.15.162:6801/572103448] [v2:172.21.15.162:6802/572103448,v1:172.21.15.162:6803/572103448] exists,up aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 24 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/2620350256,v1:172.21.15.45:6811/2620350256] [v2:172.21.15.45:6812/2620350256,v1:172.21.15.45:6813/2620350256] exists,up 14f9d3be-6d90-4490-b946-4a96c46736f9 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6808/2470611139,v1:172.21.15.47:6809/2470611139] [v2:172.21.15.47:6810/2470611139,v1:172.21.15.47:6811/2470611139] exists,up 3e9ec487-151e-4982-b522-3a2c3f90e407 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 37 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6808/2110302929,v1:172.21.15.162:6809/2110302929] [v2:172.21.15.162:6810/2110302929,v1:172.21.15.162:6811/2110302929] exists,up 086c844a-fd5a-4ded-9156-a2ccd034521f 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 44 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/135941807,v1:172.21.15.45:6819/135941807] [v2:172.21.15.45:6820/135941807,v1:172.21.15.45:6821/135941807] exists,up 29470075-95e2-48d8-a4f3-88b0b390d2a8 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 49 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6816/2101068416,v1:172.21.15.47:6817/2101068416] [v2:172.21.15.47:6818/2101068416,v1:172.21.15.47:6819/2101068416] exists,up 32004f02-5b68-43bf-9af9-f9cabebe8043 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 56 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6816/3586576942,v1:172.21.15.162:6817/3586576942] [v2:172.21.15.162:6818/3586576942,v1:172.21.15.162:6819/3586576942] exists,up 3e8cc6aa-174a-4685-a735-e8a9aed57f04 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/4106016815,v1:172.21.15.45:6827/4106016815] [v2:172.21.15.45:6828/4106016815,v1:172.21.15.45:6829/4106016815] exists,up 12ab7cd4-9ec8-46b4-9a89-5d954e80b692 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 68 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6824/3408198504,v1:172.21.15.47:6825/3408198504] [v2:172.21.15.47:6826/3408198504,v1:172.21.15.47:6827/3408198504] exists,up eb823514-17df-4818-bfdf-7b8cdbcfbcd4 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.162:6824/1692831932,v1:172.21.15.162:6825/1692831932] [v2:172.21.15.162:6826/1692831932,v1:172.21.15.162:6827/1692831932] exists,up 147281ba-9297-46a4-b3a1-c12eaa3456b3 2024-09-10T10:02:58.134 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.8 [2,11] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1d [8,11] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.5 [3,9] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.e [5,9] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.14 [4,6] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.17 [4,7] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.27 [4,8] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.37 [1,9] 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/4168396686 expires 2024-09-11T09:54:48.533581+0000 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/1958743370 expires 2024-09-11T09:54:48.533581+0000 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/1958743370 expires 2024-09-11T09:54:48.533581+0000 2024-09-10T10:02:58.135 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2521990938 expires 2024-09-11T09:54:48.533581+0000 2024-09-10T10:02:58.136 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/986599852 expires 2024-09-11T09:54:24.047197+0000 2024-09-10T10:02:58.136 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/986599852 expires 2024-09-11T09:54:24.047197+0000 2024-09-10T10:02:58.136 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/444500118 expires 2024-09-11T09:54:48.533581+0000 2024-09-10T10:02:58.136 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1519065487 expires 2024-09-11T09:54:24.047197+0000 2024-09-10T10:02:58.136 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1473601885 expires 2024-09-11T09:54:24.047197+0000 2024-09-10T10:02:58.136 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3534138672 expires 2024-09-11T09:54:24.047197+0000 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f57380a2c10 msgr2=0x7f57380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f57380a2c10 0x7f57380a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f574c101b70 tx=0x7f574800a590 comp rx=0 tx=0).stop 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 msgr2=0x7f574c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c19cc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f573c009230 tx=0x7f573c004430 comp rx=0 tx=0).stop 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 shutdown_connections 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f57380a2c10 0x7f57380a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f574c1026d0 0x7f574c19f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f574c101d10 0x7f574c19cc00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 --2- 172.21.15.45:0/3153407213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f574c100b10 0x7f574c06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.131+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 >> 172.21.15.45:0/3153407213 conn(0x7f574c0fc2a0 msgr2=0x7f574c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:58.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.132+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 shutdown_connections 2024-09-10T10:02:58.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:58.132+0000 7f57535ea640 1 -- 172.21.15.45:0/3153407213 wait complete. 2024-09-10T10:02:58.811 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:58 smithi047 ceph-mon[35244]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:58.811 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:02:58 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3153407213' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:02:58.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:58 smithi045 ceph-mon[31114]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:58.818 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:02:58 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3153407213' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:02:58.848 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T10:02:58.859 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:58.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:58 smithi162 ceph-mon[34963]: pgmap v317: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:02:58.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:02:58 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3153407213' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T10:02:59.125 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:59.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 -- 172.21.15.45:0/858283357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 msgr2=0x7f7fd810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 --2- 172.21.15.45:0/858283357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd810eaa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009ca0 tx=0x7f7fd002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 -- 172.21.15.45:0/858283357 shutdown_connections 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 --2- 172.21.15.45:0/858283357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd810eaa0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 --2- 172.21.15.45:0/858283357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fd81027a0 0x7f7fd81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 --2- 172.21.15.45:0/858283357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd8101df0 0x7f7fd81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 -- 172.21.15.45:0/858283357 >> 172.21.15.45:0/858283357 conn(0x7f7fd80fd8f0 msgr2=0x7f7fd80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 -- 172.21.15.45:0/858283357 shutdown_connections 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 -- 172.21.15.45:0/858283357 wait complete. 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 Processor -- start 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.619+0000 7f7fde93a640 1 -- start start 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fde93a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd8101df0 0x7f7fd81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fde93a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fd81027a0 0x7f7fd81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fde93a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fde93a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8111fd0 con 0x7f7fd8106f20 2024-09-10T10:02:59.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fde93a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8111e50 con 0x7f7fd81027a0 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fde93a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8112150 con 0x7f7fd8101df0 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44110/0 (socket says 172.21.15.45:44110) 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 -- 172.21.15.45:0/1551756903 learned_addr learned my addr 172.21.15.45:0/1551756903 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fd77fe640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fd81027a0 0x7f7fd81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fd7fff640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd8101df0 0x7f7fd81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 -- 172.21.15.45:0/1551756903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd8101df0 msgr2=0x7f7fd81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd8101df0 0x7f7fd81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 -- 172.21.15.45:0/1551756903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fd81027a0 msgr2=0x7f7fd81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fd81027a0 0x7f7fd81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 -- 172.21.15.45:0/1551756903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd819d360 con 0x7f7fd8106f20 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fd7fff640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd8101df0 0x7f7fd81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:59.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.620+0000 7f7fdced7640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd819cc80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0003f10 tx=0x7f7fd0003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.621+0000 7f7fd57fa640 1 -- 172.21.15.45:0/1551756903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0031710 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.621+0000 7f7fd57fa640 1 -- 172.21.15.45:0/1551756903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd00318b0 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.621+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd819d5f0 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.621+0000 7f7fd57fa640 1 -- 172.21.15.45:0/1551756903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd002fb50 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.621+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd81a4f30 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.622+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd81021d0 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.623+0000 7f7fd57fa640 1 -- 172.21.15.45:0/1551756903 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7fd002fd30 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.623+0000 7f7fd57fa640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7fc00a2c10 0x7f7fc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.623+0000 7f7fd7fff640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7fc00a2c10 0x7f7fc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.624+0000 7f7fd57fa640 1 -- 172.21.15.45:0/1551756903 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7fd00f07f0 con 0x7f7fd8106f20 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.624+0000 7f7fd7fff640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7fc00a2c10 0x7f7fc00a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009b40 tx=0x7f7fc8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:59.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.627+0000 7f7fd57fa640 1 -- 172.21.15.45:0/1551756903 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fd00bd710 con 0x7f7fd8106f20 2024-09-10T10:02:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.837+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fd810a1c0 con 0x7f7fd8106f20 2024-09-10T10:02:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.838+0000 7f7fd57fa640 1 -- 172.21.15.45:0/1551756903 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11843 (secure 0 0 0) 0x7f7fd00bd010 con 0x7f7fd8106f20 2024-09-10T10:02:59.840 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:02:59.840 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"btime":"2024-09-10T10:01:44:271138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:01:43.932833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:02:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-09-10T10:02:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.840+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7fc00a2c10 msgr2=0x7f7fc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.840+0000 7f7fde93a640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7fc00a2c10 0x7f7fc00a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009b40 tx=0x7f7fc8005e50 comp rx=0 tx=0).stop 2024-09-10T10:02:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.840+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 msgr2=0x7f7fd819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.840+0000 7f7fde93a640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd819cc80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0003f10 tx=0x7f7fd0003f40 comp rx=0 tx=0).stop 2024-09-10T10:02:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.840+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 shutdown_connections 2024-09-10T10:02:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.840+0000 7f7fde93a640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f7fc00a2c10 0x7f7fc00a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.840+0000 7f7fde93a640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fd8106f20 0x7f7fd819cc80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.841+0000 7f7fde93a640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7fd81027a0 0x7f7fd81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.841+0000 7f7fde93a640 1 --2- 172.21.15.45:0/1551756903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7fd8101df0 0x7f7fd81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.841+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 >> 172.21.15.45:0/1551756903 conn(0x7f7fd80fd8f0 msgr2=0x7f7fd80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.841+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 shutdown_connections 2024-09-10T10:02:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:02:59.841+0000 7f7fde93a640 1 -- 172.21.15.45:0/1551756903 wait complete. 2024-09-10T10:03:00.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:00.819 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:00.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:00 smithi162 ceph-mon[34963]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:03:00.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:00 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1551756903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:00.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:00 smithi045 ceph-mon[31114]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:03:00.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:00 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1551756903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:00.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:00 smithi047 ceph-mon[35244]: pgmap v318: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:03:00.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:00 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1551756903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:01.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 -- 172.21.15.45:0/3424297231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 msgr2=0x7f197810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 --2- 172.21.15.45:0/3424297231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f197810d950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1970009cd0 tx=0x7f197002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 -- 172.21.15.45:0/3424297231 shutdown_connections 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 --2- 172.21.15.45:0/3424297231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f197810d950 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 --2- 172.21.15.45:0/3424297231 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1978100a80 0x7f1978100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 --2- 172.21.15.45:0/3424297231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978106a80 0x7f1978106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 -- 172.21.15.45:0/3424297231 >> 172.21.15.45:0/3424297231 conn(0x7f19780fc7a0 msgr2=0x7f19780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 -- 172.21.15.45:0/3424297231 shutdown_connections 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.259+0000 7f197ea20640 1 -- 172.21.15.45:0/3424297231 wait complete. 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.260+0000 7f197ea20640 1 Processor -- start 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.260+0000 7f197ea20640 1 -- start start 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197ea20640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1978100a80 0x7f1978198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197ea20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f1978198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197ea20640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978106a80 0x7f197819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197ea20640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1978110f20 con 0x7f1978101420 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197ea20640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1978110da0 con 0x7f1978106a80 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197ea20640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f19781110a0 con 0x7f1978100a80 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f1978198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f1978198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44130/0 (socket says 172.21.15.45:44130) 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 -- 172.21.15.45:0/2597717730 learned_addr learned my addr 172.21.15.45:0/2597717730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197cfbd640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978106a80 0x7f197819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f1977fff640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1978100a80 0x7f1978198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 -- 172.21.15.45:0/2597717730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1978100a80 msgr2=0x7f1978198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1978100a80 0x7f1978198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 -- 172.21.15.45:0/2597717730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978106a80 msgr2=0x7f197819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978106a80 0x7f197819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 -- 172.21.15.45:0/2597717730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197819d530 con 0x7f1978101420 2024-09-10T10:03:01.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f197cfbd640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978106a80 0x7f197819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.261+0000 7f19777fe640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f1978198ac0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f196c00b4a0 tx=0x7f196c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.262+0000 7f19757fa640 1 -- 172.21.15.45:0/2597717730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c014070 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.262+0000 7f19757fa640 1 -- 172.21.15.45:0/2597717730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f196c010af0 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.262+0000 7f19757fa640 1 -- 172.21.15.45:0/2597717730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c017d10 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.262+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197819d820 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.262+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19781a5100 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.263+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1978100f60 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.264+0000 7f19757fa640 1 -- 172.21.15.45:0/2597717730 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f196c017070 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.265+0000 7f19757fa640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19600a2c10 0x7f19600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.265+0000 7f19757fa640 1 -- 172.21.15.45:0/2597717730 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f196c0ce5a0 con 0x7f1978101420 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.265+0000 7f1977fff640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19600a2c10 0x7f19600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.265+0000 7f1977fff640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19600a2c10 0x7f19600a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1968009890 tx=0x7f1968008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:01.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.269+0000 7f19757fa640 1 -- 172.21.15.45:0/2597717730 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f196c09b4c0 con 0x7f1978101420 2024-09-10T10:03:01.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.482+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f197802d7c0 con 0x7f1978101420 2024-09-10T10:03:01.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.483+0000 7f19757fa640 1 -- 172.21.15.45:0/2597717730 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11843 (secure 0 0 0) 0x7f196c09adc0 con 0x7f1978101420 2024-09-10T10:03:01.484 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:03:01.484 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"btime":"2024-09-10T10:01:44:271138+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:01:43.932833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2065513838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2065513838},{"type":"v1","addr":"172.21.15.162:6839","nonce":2065513838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:03:01.484 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-09-10T10:03:01.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19600a2c10 msgr2=0x7f19600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19600a2c10 0x7f19600a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1968009890 tx=0x7f1968008040 comp rx=0 tx=0).stop 2024-09-10T10:03:01.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 msgr2=0x7f1978198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:01.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f1978198ac0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f196c00b4a0 tx=0x7f196c00b960 comp rx=0 tx=0).stop 2024-09-10T10:03:01.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 shutdown_connections 2024-09-10T10:03:01.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f19600a2c10 0x7f19600a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978106a80 0x7f197819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1978101420 0x7f1978198ac0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 --2- 172.21.15.45:0/2597717730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1978100a80 0x7f1978198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:01.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 >> 172.21.15.45:0/2597717730 conn(0x7f19780fc7a0 msgr2=0x7f197806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:01.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 shutdown_connections 2024-09-10T10:03:01.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:01.485+0000 7f197ea20640 1 -- 172.21.15.45:0/2597717730 wait complete. 2024-09-10T10:03:02.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:03:02.449 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:02.785 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:02 smithi045 ceph-mon[31114]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:02.786 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:02 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2597717730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 -- 172.21.15.45:0/2836724611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458102790 msgr2=0x7f74581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 --2- 172.21.15.45:0/2836724611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458102790 0x7f74581067e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c02a410 comp rx=0 tx=0).stop 2024-09-10T10:03:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 -- 172.21.15.45:0/2836724611 shutdown_connections 2024-09-10T10:03:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 --2- 172.21.15.45:0/2836724611 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7458106f10 0x7f745810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 --2- 172.21.15.45:0/2836724611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458102790 0x7f74581067e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 --2- 172.21.15.45:0/2836724611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7458101de0 0x7f74581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 -- 172.21.15.45:0/2836724611 >> 172.21.15.45:0/2836724611 conn(0x7f74580fd8c0 msgr2=0x7f74580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 -- 172.21.15.45:0/2836724611 shutdown_connections 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.866+0000 7f745cdf1640 1 -- 172.21.15.45:0/2836724611 wait complete. 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 Processor -- start 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 -- start start 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7458101de0 0x7f74581984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7458102790 0x7f7458198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458106f10 0x7f745819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7458111f70 con 0x7f7458106f10 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7458111df0 con 0x7f7458102790 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f745cdf1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f74581120f0 con 0x7f7458101de0 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f7456d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458106f10 0x7f745819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.867+0000 7f7456d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458106f10 0x7f745819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44150/0 (socket says 172.21.15.45:44150) 2024-09-10T10:03:02.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7456d9d640 1 -- 172.21.15.45:0/3020262525 learned_addr learned my addr 172.21.15.45:0/3020262525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7455d9b640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7458102790 0x7f7458198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f745659c640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7458101de0 0x7f74581984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7456d9d640 1 -- 172.21.15.45:0/3020262525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7458101de0 msgr2=0x7f74581984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7456d9d640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7458101de0 0x7f74581984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7456d9d640 1 -- 172.21.15.45:0/3020262525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7458102790 msgr2=0x7f7458198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7456d9d640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7458102790 0x7f7458198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7456d9d640 1 -- 172.21.15.45:0/3020262525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745819d470 con 0x7f7458106f10 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f745659c640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7458101de0 0x7f74581984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7455d9b640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7458102790 0x7f7458198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f7456d9d640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458106f10 0x7f745819cd90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f745000cad0 tx=0x7f745000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f74477fe640 1 -- 172.21.15.45:0/3020262525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745000ae20 con 0x7f7458106f10 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f74477fe640 1 -- 172.21.15.45:0/3020262525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7450005020 con 0x7f7458106f10 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745819d760 con 0x7f7458106f10 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f74477fe640 1 -- 172.21.15.45:0/3020262525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745000f6f0 con 0x7f7458106f10 2024-09-10T10:03:02.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.868+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581a5040 con 0x7f7458106f10 2024-09-10T10:03:02.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.869+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74581021c0 con 0x7f7458106f10 2024-09-10T10:03:02.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.870+0000 7f74477fe640 1 -- 172.21.15.45:0/3020262525 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f74500051d0 con 0x7f7458106f10 2024-09-10T10:03:02.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.871+0000 7f74477fe640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74400a2c90 0x7f74400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:02.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.871+0000 7f745659c640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74400a2c90 0x7f74400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:02.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.871+0000 7f74477fe640 1 -- 172.21.15.45:0/3020262525 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f74500cf560 con 0x7f7458106f10 2024-09-10T10:03:02.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.871+0000 7f745659c640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74400a2c90 0x7f74400a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74480058c0 tx=0x7f7448005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:02.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:02.874+0000 7f74477fe640 1 -- 172.21.15.45:0/3020262525 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7450014030 con 0x7f7458106f10 2024-09-10T10:03:02.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:02 smithi162 ceph-mon[34963]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:02.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:02 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2597717730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:02.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:02 smithi047 ceph-mon[35244]: pgmap v319: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:02.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:02 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2597717730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:03.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.055+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7458199820 con 0x7f7458106f10 2024-09-10T10:03:03.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.056+0000 7f74477fe640 1 -- 172.21.15.45:0/3020262525 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22675 (secure 0 0 0) 0x7f7458199820 con 0x7f7458106f10 2024-09-10T10:03:03.058 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:03:03.058 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:01:50.297428+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:03:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74400a2c90 msgr2=0x7f74400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74400a2c90 0x7f74400a5150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f74480058c0 tx=0x7f7448005d30 comp rx=0 tx=0).stop 2024-09-10T10:03:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458106f10 msgr2=0x7f745819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:03.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458106f10 0x7f745819cd90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f745000cad0 tx=0x7f745000cf90 comp rx=0 tx=0).stop 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 shutdown_connections 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f74400a2c90 0x7f74400a5150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7458106f10 0x7f745819cd90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7458102790 0x7f7458198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 --2- 172.21.15.45:0/3020262525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7458101de0 0x7f74581984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 >> 172.21.15.45:0/3020262525 conn(0x7f74580fd8c0 msgr2=0x7f74580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 shutdown_connections 2024-09-10T10:03:03.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:03.059+0000 7f745cdf1640 1 -- 172.21.15.45:0/3020262525 wait complete. 2024-09-10T10:03:03.765 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:03 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3020262525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:03:03.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T10:01:34.924889+0000', 'modified': '2024-09-10T10:01:43.932833+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.162:6839/2065513838', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.162:6838', 'nonce': 2065513838}, {'type': 'v1', 'addr': '172.21.15.162:6839', 'nonce': 2065513838}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6839/2511396298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6838', 'nonce': 2511396298}, {'type': 'v1', 'addr': '172.21.15.47:6839', 'nonce': 2511396298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24488]} 2024-09-10T10:03:03.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:03:03.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T10:03:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:03 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3020262525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:03:03.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:03 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3020262525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:03:04.059 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2456394094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a410c170 msgr2=0x7f66a410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2456394094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a410c170 0x7f66a410e610 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f669c009ca0 tx=0x7f669c02a950 comp rx=0 tx=0).stop 2024-09-10T10:03:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2456394094 shutdown_connections 2024-09-10T10:03:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2456394094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a410c170 0x7f66a410e610 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2456394094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a4103660 0x7f66a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2456394094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66a4102d40 0x7f66a4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2456394094 >> 172.21.15.45:0/2456394094 conn(0x7f66a40fc7a0 msgr2=0x7f66a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2456394094 shutdown_connections 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.519+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2456394094 wait complete. 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 Processor -- start 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 -- start start 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66a4102d40 0x7f66a41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a4103660 0x7f66a4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a410c170 0x7f66a419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4111f70 con 0x7f66a4103660 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4111df0 con 0x7f66a410c170 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.520+0000 7f66aa1fa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41120f0 con 0x7f66a4102d40 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a4103660 0x7f66a4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a4103660 0x7f66a4198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49242/0 (socket says 172.21.15.45:49242) 2024-09-10T10:03:04.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 -- 172.21.15.45:0/2422954148 learned_addr learned my addr 172.21.15.45:0/2422954148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a37fe640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66a4102d40 0x7f66a41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 -- 172.21.15.45:0/2422954148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66a4102d40 msgr2=0x7f66a41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66a4102d40 0x7f66a41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 -- 172.21.15.45:0/2422954148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a410c170 msgr2=0x7f66a419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a3fff640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a410c170 0x7f66a419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a410c170 0x7f66a419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 -- 172.21.15.45:0/2422954148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a419d3a0 con 0x7f66a4103660 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a4103660 0x7f66a4198930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f669800c970 tx=0x7f669800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a0ff9640 1 -- 172.21.15.45:0/2422954148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698007bb0 con 0x7f66a4103660 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a0ff9640 1 -- 172.21.15.45:0/2422954148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6698007d50 con 0x7f66a4103660 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a419d690 con 0x7f66a4103660 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66a0ff9640 1 -- 172.21.15.45:0/2422954148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698018820 con 0x7f66a4103660 2024-09-10T10:03:04.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.521+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a41a4f70 con 0x7f66a4103660 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.524+0000 7f66a0ff9640 1 -- 172.21.15.45:0/2422954148 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f66980189c0 con 0x7f66a4103660 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.524+0000 7f66ac5d3740 1 Processor -- start 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.524+0000 7f66ac5d3740 1 -- start start 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d858090 con 0x55bc3d8f19d0 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d7ffa70 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d854050 con 0x55bc3d8f22f0 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49244/0 (socket says 172.21.15.45:49244) 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 -- 172.21.15.45:0/761190848 learned_addr learned my addr 172.21.15.45:0/761190848 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a37fe640 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a0ff9640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f668c0a2c10 0x7f668c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 -- 172.21.15.45:0/761190848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 msgr2=0x55bc3d8fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 -- 172.21.15.45:0/761190848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 msgr2=0x55bc3d8fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a3fff640 1 -- 172.21.15.45:0/761190848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc3d80c130 con 0x55bc3d8f19d0 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.525+0000 7f66a0ff9640 1 -- 172.21.15.45:0/2422954148 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f66980cf280 con 0x7f66a4103660 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.526+0000 7f66a3fff640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f668c0a2c10 0x7f668c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.526+0000 7f66a3fff640 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8f1db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f669c009930 tx=0x7f669c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c815c877fbe4b86a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.526+0000 7f6689ffb640 1 -- 172.21.15.45:0/761190848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c034050 con 0x55bc3d8f19d0 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.526+0000 7f6689ffb640 1 -- 172.21.15.45:0/761190848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c02f6a0 con 0x55bc3d8f19d0 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.526+0000 7f6689ffb640 1 -- 172.21.15.45:0/761190848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c030790 con 0x55bc3d8f19d0 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 -- 172.21.15.45:0/761190848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 msgr2=0x55bc3d8f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8f1db0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f669c009930 tx=0x7f669c009960 comp rx=0 tx=0).stop 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66a3fff640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f668c0a2c10 0x7f668c0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f669c048000 tx=0x7f669c049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 -- 172.21.15.45:0/761190848 shutdown_connections 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/761190848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8f1db0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 -- 172.21.15.45:0/761190848 >> 172.21.15.45:0/761190848 conn(0x55bc3d8edf60 msgr2=0x55bc3d8ee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 -- 172.21.15.45:0/761190848 shutdown_connections 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 -- 172.21.15.45:0/761190848 wait complete. 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.527+0000 7f66ac5d3740 1 -- start start 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 1 Processor -- start 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8fc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d8ff780 con 0x55bc3d8f22f0 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d812f20 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d8ff900 con 0x55bc3d8f19d0 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66ac5d3740 10 client.0 _ll_register_callbacks cb 0x55bc3d7dc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:57494/0 (socket says 172.21.15.45:57494) 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49246/0 (socket says 172.21.15.45:49246) 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 -- 172.21.15.45:0/2527424956 learned_addr learned my addr 172.21.15.45:0/2527424956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a37fe640 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 -- 172.21.15.45:0/2527424956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f19d0 msgr2=0x55bc3d8fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 -- 172.21.15.45:0/2527424956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f22f0 msgr2=0x55bc3d8f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a3fff640 1 -- 172.21.15.45:0/2527424956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc3d5c92b0 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.528+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f66a3fff640 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8f9ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f669c030900 tx=0x7f669c030c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e87c8e7f70f02ff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f6682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f6682ffd640 1 -- 172.21.15.45:0/2527424956 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c034070 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f6682ffd640 1 -- 172.21.15.45:0/2527424956 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f669c003f00 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c02f6a0 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f6682ffd640 1 -- 172.21.15.45:0/2527424956 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f669c02fe10 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f66ac5d3740 10 client.24800 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bc3d8ff780 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.529+0000 7f6682ffd640 1 -- 172.21.15.45:0/2527424956 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669c02f6a0 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f6682ffd640 1 -- 172.21.15.45:0/2527424956 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f669c050640 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66ac5d3740 10 client.24800 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:04.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66ac5d3740 10 client.24800 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66ac5d3740 20 client.24800 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d90d280 0x55bc3d90e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66ac5d3740 4 client.24800 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc3d865600 con 0x55bc3d90d280 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66a37fe640 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d90d280 0x55bc3d90e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f66a37fe640 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d90d280 0x55bc3d90e6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e9df4d6cdf37e44 server_cookie=2c75e3ccada62f84 in_seq=0 out_seq=0 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.530+0000 7f6682ffd640 10 client.24800 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f6682ffd640 1 -- 172.21.15.45:0/2527424956 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc3d865600 con 0x55bc3d90d280 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f6682ffd640 10 client.24800 handle_command_reply: tid=0 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 client.24800 shutdown 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 msgr2=0x55bc3d8f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8f9ca0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f669c030900 tx=0x7f669c030c70 comp rx=0 tx=0).stop 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 shutdown_connections 2024-09-10T10:03:04.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d90d280 0x55bc3d90e6f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8f9ca0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f22f0 0x55bc3d8f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f6682ffd640 0 client.24800 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/2527424956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f19d0 0x55bc3d8fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 >> 172.21.15.45:0/2527424956 conn(0x55bc3d8edf60 msgr2=0x55bc3d8f3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.531+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 shutdown_connections 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.532+0000 7f66ac5d3740 1 -- 172.21.15.45:0/2527424956 wait complete. 2024-09-10T10:03:04.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.532+0000 7f66ac5d3740 20 client.24800 trim_cache size 0 max 16384 2024-09-10T10:03:04.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.549+0000 7f66ac5d3740 1 Processor -- start 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.549+0000 7f66ac5d3740 1 -- start start 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d90e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d90c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8ecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d90ca30 con 0x55bc3d8f19d0 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d90d100 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d90d7d0 con 0x55bc3d8f22f0 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d90e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d90e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49248/0 (socket says 172.21.15.45:49248) 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 -- 172.21.15.45:0/3486811585 learned_addr learned my addr 172.21.15.45:0/3486811585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d90c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 -- 172.21.15.45:0/3486811585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 msgr2=0x55bc3d90c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d90c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 -- 172.21.15.45:0/3486811585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 msgr2=0x55bc3d8ecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8ecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 -- 172.21.15.45:0/3486811585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc3d8f1800 con 0x55bc3d8f19d0 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d90c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66a37fe640 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d90e6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f669400c350 tx=0x7f6694012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52c455dc64e9253 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66897fa640 1 -- 172.21.15.45:0/3486811585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694004020 con 0x55bc3d8f19d0 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66897fa640 1 -- 172.21.15.45:0/3486811585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66940041c0 con 0x55bc3d8f19d0 2024-09-10T10:03:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.550+0000 7f66897fa640 1 -- 172.21.15.45:0/3486811585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694010690 con 0x55bc3d8f19d0 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 -- 172.21.15.45:0/3486811585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 msgr2=0x55bc3d90e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d90e6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f669400c350 tx=0x7f6694012820 comp rx=0 tx=0).stop 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 -- 172.21.15.45:0/3486811585 shutdown_connections 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d8ecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8f22f0 0x55bc3d90c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/3486811585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d90e6f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 -- 172.21.15.45:0/3486811585 >> 172.21.15.45:0/3486811585 conn(0x55bc3d8edf60 msgr2=0x55bc3d8ec800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 -- 172.21.15.45:0/3486811585 shutdown_connections 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 -- 172.21.15.45:0/3486811585 wait complete. 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.551+0000 7f66ac5d3740 1 -- start start 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 1 Processor -- start 2024-09-10T10:03:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d916cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8f22f0 0x55bc3d917200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d912080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d90e570 con 0x55bc3d8f19d0 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d90dea0 con 0x55bc3d8f22f0 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55bc3d90ec40 con 0x55bc3d8fcbb0 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66ac5d3740 10 client.0 _ll_register_callbacks cb 0x7f669c047020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d916cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d916cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49254/0 (socket says 172.21.15.45:49254) 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 -- 172.21.15.45:0/1410375892 learned_addr learned my addr 172.21.15.45:0/1410375892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8f22f0 0x55bc3d917200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a3fff640 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d912080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 -- 172.21.15.45:0/1410375892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8fcbb0 msgr2=0x55bc3d912080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d912080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 -- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8f22f0 msgr2=0x55bc3d917200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8f22f0 0x55bc3d917200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.552+0000 7f66a37fe640 1 -- 172.21.15.45:0/1410375892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc3d9128b0 con 0x55bc3d8f19d0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f66a37fe640 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d916cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6694003ec0 tx=0x7f66940045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ab9ceec2ed2108 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f6682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f6682ffd640 1 -- 172.21.15.45:0/1410375892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669402a070 con 0x55bc3d8f19d0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f6682ffd640 1 -- 172.21.15.45:0/1410375892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6694003a00 con 0x55bc3d8f19d0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66940041c0 con 0x55bc3d8f19d0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f6682ffd640 1 -- 172.21.15.45:0/1410375892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6694036050 con 0x55bc3d8f19d0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f66ac5d3740 10 client.15036 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55bc3d9774c0 con 0x55bc3d8f19d0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f6682ffd640 1 -- 172.21.15.45:0/1410375892 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66940041c0 con 0x55bc3d8f19d0 2024-09-10T10:03:04.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f6682ffd640 1 -- 172.21.15.45:0/1410375892 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6694026440 con 0x55bc3d8f19d0 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.553+0000 7f66ac5d3740 10 client.15036 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66ac5d3740 10 client.15036 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66ac5d3740 20 client.15036 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d97bd40 0x55bc3d97aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66ac5d3740 4 client.15036 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "status"}) -- 0x55bc3d85fb10 con 0x55bc3d97bd40 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d97bd40 0x55bc3d97aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66a2ffd640 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d97bd40 0x55bc3d97aa50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1914ca08e40290 server_cookie=ae7ae29bd17e1542 in_seq=0 out_seq=0 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f6682ffd640 10 client.15036 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f6682ffd640 1 -- 172.21.15.45:0/1410375892 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bc3d85fb10 con 0x55bc3d97bd40 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f6682ffd640 10 client.15036 handle_command_reply: tid=0 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.554+0000 7f66ac5d3740 1 client.15036 shutdown 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 msgr2=0x55bc3d916cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d916cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6694003ec0 tx=0x7f66940045a0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 shutdown_connections 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55bc3d97bd40 0x55bc3d97aa50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55bc3d8fcbb0 0x55bc3d912080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bc3d8f22f0 0x55bc3d917200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f6682ffd640 0 client.15036 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 --2- 172.21.15.45:0/1410375892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3d8f19d0 0x55bc3d916cc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 >> 172.21.15.45:0/1410375892 conn(0x55bc3d8edf60 msgr2=0x55bc3d8ed020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 shutdown_connections 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 1 -- 172.21.15.45:0/1410375892 wait complete. 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66ac5d3740 20 client.15036 trim_cache size 0 max 16384 2024-09-10T10:03:04.561 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "729b10fe-6f5a-11ef-bcea-c7b262605968", 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24488, 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 89.317076169000003, 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 19, 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 83, 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 81, 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 100.204295323 2024-09-10T10:03:04.562 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T10:03:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f668c0a2c10 msgr2=0x7f668c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f668c0a2c10 0x7f668c0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f669c048000 tx=0x7f669c049000 comp rx=0 tx=0).stop 2024-09-10T10:03:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a4103660 msgr2=0x7f66a4198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.555+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a4103660 0x7f66a4198930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f669800c970 tx=0x7f669800ce30 comp rx=0 tx=0).stop 2024-09-10T10:03:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 shutdown_connections 2024-09-10T10:03:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7f668c0a2c10 0x7f668c0a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66a410c170 0x7f66a419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66a4103660 0x7f66a4198930 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 --2- 172.21.15.45:0/2422954148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f66a4102d40 0x7f66a41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 >> 172.21.15.45:0/2422954148 conn(0x7f66a40fc7a0 msgr2=0x7f66a40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 shutdown_connections 2024-09-10T10:03:04.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:04.556+0000 7f66aa1fa640 1 -- 172.21.15.45:0/2422954148 wait complete. 2024-09-10T10:03:04.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:04 smithi162 ceph-mon[34963]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:03:04.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:04 smithi045 ceph-mon[31114]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:03:04.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:04 smithi047 ceph-mon[35244]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T10:03:05.349 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T10:03:05.358 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T10:03:05.358 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T10:03:05.359 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T10:03:05.359 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-10T10:03:05.360 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T10:03:05.360 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:03:05.360 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-10T10:03:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.466+0000 7fea52dc0640 1 -- 172.21.15.45:0/120462801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1013d0 msgr2=0x7fea4c101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.466+0000 7fea52dc0640 1 --2- 172.21.15.45:0/120462801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1013d0 0x7fea4c101850 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fea38009900 tx=0x7fea3802a410 comp rx=0 tx=0).stop 2024-09-10T10:03:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.466+0000 7fea52dc0640 1 -- 172.21.15.45:0/120462801 shutdown_connections 2024-09-10T10:03:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.466+0000 7fea52dc0640 1 --2- 172.21.15.45:0/120462801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c101d90 0x7fea4c108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.466+0000 7fea52dc0640 1 --2- 172.21.15.45:0/120462801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1013d0 0x7fea4c101850 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.466+0000 7fea52dc0640 1 --2- 172.21.15.45:0/120462801 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c1001e0 0x7fea4c1005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.466+0000 7fea52dc0640 1 -- 172.21.15.45:0/120462801 >> 172.21.15.45:0/120462801 conn(0x7fea4c0fb980 msgr2=0x7fea4c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.467+0000 7fea52dc0640 1 -- 172.21.15.45:0/120462801 shutdown_connections 2024-09-10T10:03:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.467+0000 7fea52dc0640 1 -- 172.21.15.45:0/120462801 wait complete. 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.467+0000 7fea52dc0640 1 Processor -- start 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.467+0000 7fea52dc0640 1 -- start start 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea52dc0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1001e0 0x7fea4c195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea52dc0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c1013d0 0x7fea4c196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea52dc0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c101d90 0x7fea4c19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea52dc0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c10b610 con 0x7fea4c1001e0 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea52dc0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c10b490 con 0x7fea4c1013d0 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea52dc0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c10b790 con 0x7fea4c101d90 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1001e0 0x7fea4c195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1001e0 0x7fea4c195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49282/0 (socket says 172.21.15.45:49282) 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 -- 172.21.15.45:0/13030834 learned_addr learned my addr 172.21.15.45:0/13030834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea5135d640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c101d90 0x7fea4c19d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea4bfff640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c1013d0 0x7fea4c196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 -- 172.21.15.45:0/13030834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c101d90 msgr2=0x7fea4c19d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c101d90 0x7fea4c19d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 -- 172.21.15.45:0/13030834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c1013d0 msgr2=0x7fea4c196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c1013d0 0x7fea4c196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 -- 172.21.15.45:0/13030834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c19d7f0 con 0x7fea4c1001e0 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea4bfff640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c1013d0 0x7fea4c196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea50b5c640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1001e0 0x7fea4c195bd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fea4000eba0 tx=0x7fea4000a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea49ffb640 1 -- 172.21.15.45:0/13030834 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40004020 con 0x7fea4c1001e0 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c19dae0 con 0x7fea4c1001e0 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea49ffb640 1 -- 172.21.15.45:0/13030834 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea400041c0 con 0x7fea4c1001e0 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.468+0000 7fea49ffb640 1 -- 172.21.15.45:0/13030834 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40018680 con 0x7fea4c1001e0 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.469+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c19e030 con 0x7fea4c1001e0 2024-09-10T10:03:05.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.470+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4c107580 con 0x7fea4c1001e0 2024-09-10T10:03:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.471+0000 7fea49ffb640 1 -- 172.21.15.45:0/13030834 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea400035f0 con 0x7fea4c1001e0 2024-09-10T10:03:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.472+0000 7fea49ffb640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fea3c0a2cd0 0x7fea3c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.472+0000 7fea4bfff640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fea3c0a2cd0 0x7fea3c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.472+0000 7fea49ffb640 1 -- 172.21.15.45:0/13030834 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fea400cff30 con 0x7fea4c1001e0 2024-09-10T10:03:05.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.472+0000 7fea4bfff640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fea3c0a2cd0 0x7fea3c0a5190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea38031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:05.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.476+0000 7fea49ffb640 1 -- 172.21.15.45:0/13030834 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea40013070 con 0x7fea4c1001e0 2024-09-10T10:03:05.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:05.701+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fea4c02d990 con 0x7fea4c1001e0 2024-09-10T10:03:06.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.483+0000 7fea49ffb640 1 -- 172.21.15.45:0/13030834 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7fea4009cce0 con 0x7fea4c1001e0 2024-09-10T10:03:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fea3c0a2cd0 msgr2=0x7fea3c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:06.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fea3c0a2cd0 0x7fea3c0a5190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fea38000c00 tx=0x7fea38031400 comp rx=0 tx=0).stop 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1001e0 msgr2=0x7fea4c195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1001e0 0x7fea4c195bd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fea4000eba0 tx=0x7fea4000a720 comp rx=0 tx=0).stop 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 shutdown_connections 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fea3c0a2cd0 0x7fea3c0a5190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea4c101d90 0x7fea4c19d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea4c1013d0 0x7fea4c196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 --2- 172.21.15.45:0/13030834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea4c1001e0 0x7fea4c195bd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 >> 172.21.15.45:0/13030834 conn(0x7fea4c0fb980 msgr2=0x7fea4c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 shutdown_connections 2024-09-10T10:03:06.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:06.486+0000 7fea52dc0640 1 -- 172.21.15.45:0/13030834 wait complete. 2024-09-10T10:03:06.510 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T10:03:06.539 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T10:03:06.539 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T10:03:06.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:06.539 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-10T10:03:06.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:06.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-10T10:03:06.629 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-09-10T10:03:06.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:03:06.631 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-10T10:03:06.631 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:03:06.631 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:06.631 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T10:03:06.842 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-mon[34963]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:06.843 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/13030834' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-10T10:03:06.843 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: ignoring --setuser ceph since I am not root 2024-09-10T10:03:06.843 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: ignoring --setgroup ceph since I am not root 2024-09-10T10:03:06.843 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-mgr[36777]: -- 172.21.15.162:0/1853446240 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b1038b0e0 con 0x557b10368800 2024-09-10T10:03:06.843 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:06.635+0000 7f0d251c0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:03:06.843 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:06.692+0000 7f0d251c0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:03:06.843 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:06.841+0000 7f0d251c0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:03:06.857 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-mon[35244]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:06.858 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/13030834' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-10T10:03:06.858 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: ignoring --setuser ceph since I am not root 2024-09-10T10:03:06.858 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: ignoring --setgroup ceph since I am not root 2024-09-10T10:03:06.858 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-mgr[36610]: -- 172.21.15.47:0/1248151970 <== mon.0 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c67c8350e0 con 0x55c67c813000 2024-09-10T10:03:06.858 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:06.643+0000 7f797e749040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:03:06.858 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:06.702+0000 7f797e749040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:03:06.858 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:06.856+0000 7f797e749040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:03:06.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-mon[31114]: pgmap v321: 97 pgs: 97 active+clean; 639 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:06.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/13030834' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-10T10:03:06.874 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ignoring --setuser ceph since I am not root 2024-09-10T10:03:06.874 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ignoring --setgroup ceph since I am not root 2024-09-10T10:03:06.874 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:06.651+0000 7fc6db0c9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:03:06.874 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:06.712+0000 7fc6db0c9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:03:06.874 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:06.872+0000 7fc6db0c9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:03:06.937 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:07.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:06.907+0000 7f0d251c0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:03:07.149 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:07.054+0000 7f0d251c0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:03:07.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:06.951+0000 7fc6db0c9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:03:07.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:07.107+0000 7fc6db0c9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:03:07.159 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:06.925+0000 7f797e749040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:03:07.159 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:07.079+0000 7f797e749040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 -- 172.21.15.45:0/2499478281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 msgr2=0x7fae601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 --2- 172.21.15.45:0/2499478281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 0x7fae601029b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009900 tx=0x7fae4c02a480 comp rx=0 tx=0).stop 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 -- 172.21.15.45:0/2499478281 shutdown_connections 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 --2- 172.21.15.45:0/2499478281 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60102ef0 0x7fae6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 --2- 172.21.15.45:0/2499478281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 0x7fae601029b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 --2- 172.21.15.45:0/2499478281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 0x7fae60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 -- 172.21.15.45:0/2499478281 >> 172.21.15.45:0/2499478281 conn(0x7fae600fc7d0 msgr2=0x7fae600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 -- 172.21.15.45:0/2499478281 shutdown_connections 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.405+0000 7fae66cff640 1 -- 172.21.15.45:0/2499478281 wait complete. 2024-09-10T10:03:07.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 Processor -- start 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 -- start start 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 0x7fae60199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 0x7fae60199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60102ef0 0x7fae601a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae60110f80 con 0x7fae60102570 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae60110e00 con 0x7fae60100e20 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae66cff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fae60111100 con 0x7fae60102ef0 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 0x7fae60199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 0x7fae60199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 0x7fae60199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:57550/0 (socket says 172.21.15.45:57550) 2024-09-10T10:03:07.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 -- 172.21.15.45:0/2793577444 learned_addr learned my addr 172.21.15.45:0/2793577444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae6529c640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60102ef0 0x7fae601a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 -- 172.21.15.45:0/2793577444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60102ef0 msgr2=0x7fae601a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60102ef0 0x7fae601a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 -- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 msgr2=0x7fae60199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 0x7fae60199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.406+0000 7fae64a9b640 1 -- 172.21.15.45:0/2793577444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae601a4910 con 0x7fae60100e20 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.407+0000 7fae64a9b640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 0x7fae60199350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.407+0000 7fae5dffb640 1 -- 172.21.15.45:0/2793577444 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5400cd80 con 0x7fae60100e20 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.407+0000 7fae5dffb640 1 -- 172.21.15.45:0/2793577444 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae54010910 con 0x7fae60100e20 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.407+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae601a4ae0 con 0x7fae60100e20 2024-09-10T10:03:07.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.407+0000 7fae5dffb640 1 -- 172.21.15.45:0/2793577444 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54018990 con 0x7fae60100e20 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.407+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae601a4fa0 con 0x7fae60100e20 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.408+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae60101200 con 0x7fae60100e20 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.410+0000 7fae5dffb640 1 -- 172.21.15.45:0/2793577444 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 137901+0+0 (secure 0 0 0) 0x7fae54010ac0 con 0x7fae60100e20 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.410+0000 7fae5dffb640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 0x7fae500a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.410+0000 7fae5ffff640 1 -- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 msgr2=0x7fae500a5120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/952575454 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.410+0000 7fae5dffb640 1 -- 172.21.15.45:0/2793577444 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fae540cf9d0 con 0x7fae60100e20 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.410+0000 7fae5ffff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 0x7fae500a5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:03:07.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.413+0000 7fae5dffb640 1 -- 172.21.15.45:0/2793577444 <== mon.2 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae5409c130 con 0x7fae60100e20 2024-09-10T10:03:07.434 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:07.258+0000 7fc6db0c9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:03:07.504 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:07.195+0000 7f0d251c0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:03:07.563 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:07.226+0000 7f797e749040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:03:07.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.611+0000 7fae5ffff640 1 -- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 msgr2=0x7fae500a5120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/952575454 2024-09-10T10:03:07.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.611+0000 7fae5ffff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 0x7fae500a5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:03:07.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.632+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fae6006a560 con 0x7fae60100e20 2024-09-10T10:03:07.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.632+0000 7fae5dffb640 1 -- 172.21.15.45:0/2793577444 <== mon.2 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7fae6006a560 con 0x7fae60100e20 2024-09-10T10:03:07.634 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:03:07.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.634+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 msgr2=0x7fae500a5120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:03:07.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.634+0000 7fae66cff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 0x7fae500a5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.634+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 msgr2=0x7fae60199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 0x7fae60199350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 shutdown_connections 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:6800/952575454,v1:172.21.15.45:6801/952575454] conn(0x7fae500a2c60 0x7fae500a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae60102ef0 0x7fae601a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae60102570 0x7fae60199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 --2- 172.21.15.45:0/2793577444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae60100e20 0x7fae60199350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 >> 172.21.15.45:0/2793577444 conn(0x7fae600fc7d0 msgr2=0x7fae601068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 shutdown_connections 2024-09-10T10:03:07.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:07.635+0000 7fae66cff640 1 -- 172.21.15.45:0/2793577444 wait complete. 2024-09-10T10:03:07.816 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:07 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/13030834' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T10:03:07.817 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:07 smithi047 ceph-mon[35244]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-10T10:03:07.817 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:07.556+0000 7f797e749040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:03:07.817 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:07.620+0000 7f797e749040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:03:07.817 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:07.815+0000 7f797e749040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:03:07.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:07 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/13030834' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T10:03:07.857 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:07 smithi045 ceph-mon[31114]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-10T10:03:07.858 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:07.592+0000 7fc6db0c9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:03:07.858 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:07.659+0000 7fc6db0c9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:03:07.858 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:07.855+0000 7fc6db0c9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:03:07.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:07 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/13030834' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T10:03:07.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:07 smithi162 ceph-mon[34963]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-10T10:03:07.900 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:07.506+0000 7f0d251c0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:03:07.901 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:07.567+0000 7f0d251c0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:03:07.901 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:07.752+0000 7f0d251c0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:03:08.255 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:03:08.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:08.256 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-09-10T10:03:08.272 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:03:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:03:08.272 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-09-10T10:03:08.273 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:03:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42183sec preferred_lft 42183sec 2024-09-10T10:03:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link 2024-09-10T10:03:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:03:08.274 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:03:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T10:03:08.274 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-09-10T10:03:08.274 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:03:08.274 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:03:08.274 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-10T10:03:08.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:03:08.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-09-10T10:03:08.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-09-10T10:03:08.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:03:08.275 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-10T10:03:08.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:08.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:08.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:08.487 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:03:08.515 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-09-10T10:03:08.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:08.516 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-09-10T10:03:08.591 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-09-10T10:03:08.591 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:03:08.592 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:03:08.592 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:03:08.592 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:03:08.592 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:03:08.592 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-10T10:03:08.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:03:08.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:03:08.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:03:08.593 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-10T10:03:08.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:08.789 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:08 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2793577444' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:03:08.810 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:08 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2793577444' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:03:08.811 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:08 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:08.738+0000 7f797e749040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:03:08.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:08.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:08.878 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-10T10:03:08.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:08.895 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-09-10T10:03:08.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:08 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2793577444' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T10:03:08.899 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:08.639+0000 7f0d251c0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:03:08.900 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:08.706+0000 7f0d251c0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:03:08.952 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:03:08.952 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-10T10:03:08.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:03:08.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T10:03:08.952 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-10T10:03:09.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:09.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:08.808+0000 7fc6db0c9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:03:09.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:08.883+0000 7fc6db0c9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:03:09.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:09.073 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T10:03:09.074 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:03:09.074 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-10T10:03:09.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T10:03:09.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:03:09.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T10:03:09.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T10:03:09.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T10:03:09.075 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-10T10:03:09.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:09.159 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:08 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:08.809+0000 7f797e749040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:03:09.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:09.205 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:03:09.205 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-09-10T10:03:09.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-09-10T10:03:09.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:03:09.206 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-09-10T10:03:09.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:09.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:03:09.288 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:09.304 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:03:09.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:03:09.305 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:09.360 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T10:03:09.361 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T10:03:09.361 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/7f8e0794-462f-4a26-a474-c21ab28cee52 2024-09-10T10:03:09.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:09.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/7f8e0794-462f-4a26-a474-c21ab28cee52 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-09-10T10:03:09.457 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-09-10T10:03:09.457 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-09-10T10:03:09.458 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2024-09-10T10:03:09.601 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-09-10T10:03:09.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-09-10T10:03:09.601 INFO:teuthology.orchestra.run.smithi045.stdout:invalid new device string format 2024-09-10T10:03:09.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.47:3300,172.21.15.162:3300" 2024-09-10T10:03:09.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.45:3300,172.21.15.47:3300,172.21.15.162:3300:/volumes/qa/sv_1/7f8e0794-462f-4a26-a474-c21ab28cee52 2024-09-10T10:03:09.601 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=729b10fe-6f5a-11ef-bcea-c7b262605968" will pass to kernel 2024-09-10T10:03:09.602 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:03:09.602 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:03:09.602 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:03:09.602 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:03:09.602 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:03:09.604 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:03:09.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:03:09.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:09.674 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:09.415+0000 7f0d251c0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:03:09.674 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:09.530+0000 7f0d251c0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:03:09.781 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:09.658+0000 7fc6db0c9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:03:09.781 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:09.780+0000 7fc6db0c9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:03:09.828 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:09.555+0000 7f797e749040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:03:09.828 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:09.677+0000 7f797e749040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:03:09.952 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:09.673+0000 7f0d251c0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:03:09.952 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:09.734+0000 7f0d251c0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:03:09.953 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:09.883+0000 7f0d251c0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:03:09.953 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:09.951+0000 7f0d251c0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:03:10.120 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:09.826+0000 7f797e749040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:03:10.120 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:09.890+0000 7f797e749040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:03:10.121 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.048+0000 7f797e749040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:03:10.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:09.930+0000 7fc6db0c9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:03:10.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:09.995+0000 7fc6db0c9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:03:10.234 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:10.011+0000 7f0d251c0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:03:10.234 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:10.168+0000 7f0d251c0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:03:10.235 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:10.233+0000 7f0d251c0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:03:10.409 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.119+0000 7f797e749040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:03:10.409 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.181+0000 7f797e749040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:03:10.410 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.350+0000 7f797e749040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:03:10.457 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.152+0000 7fc6db0c9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:03:10.457 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.223+0000 7fc6db0c9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:03:10.457 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.289+0000 7fc6db0c9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:03:10.457 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.456+0000 7fc6db0c9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:03:10.507 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:10.303+0000 7f0d251c0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:03:10.507 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:10.386+0000 7f0d251c0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:03:10.507 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:10.446+0000 7f0d251c0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:03:10.507 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:10.506+0000 7f0d251c0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:03:10.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.419+0000 7f797e749040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:03:10.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.488+0000 7f797e749040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:03:10.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.575+0000 7f797e749040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:03:10.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.638+0000 7f797e749040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:03:10.701 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:10.699+0000 7f797e749040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:03:10.745 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.524+0000 7fc6db0c9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:03:10.745 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.594+0000 7fc6db0c9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:03:10.745 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.681+0000 7fc6db0c9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:03:11.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.744+0000 7fc6db0c9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:03:11.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:10.806+0000 7fc6db0c9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:03:11.646 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-mon[31114]: Standby manager daemon z restarted 2024-09-10T10:03:11.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-mon[31114]: Standby manager daemon z started 2024-09-10T10:03:11.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:03:11.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:03:11.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:03:11.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:03:11.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:11 smithi162 ceph-mon[34963]: Standby manager daemon z restarted 2024-09-10T10:03:11.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:11 smithi162 ceph-mon[34963]: Standby manager daemon z started 2024-09-10T10:03:11.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:11 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:03:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:11 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:03:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:11 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:03:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:11 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:03:11.650 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:03:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z[36772]: 2024-09-10T10:03:11.300+0000 7f0d251c0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:03:11.659 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:03:11 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y[36606]: 2024-09-10T10:03:11.534+0000 7f797e749040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:03:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:11 smithi047 ceph-mon[35244]: Standby manager daemon z restarted 2024-09-10T10:03:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:11 smithi047 ceph-mon[35244]: Standby manager daemon z started 2024-09-10T10:03:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:11 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T10:03:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:11 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:03:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:11 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T10:03:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:11 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.162:0/2173882774' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:03:11.681 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:03:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:03:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:11.754 INFO:teuthology.orchestra.run.smithi045.stdout:client24754 2024-09-10T10:03:11.754 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T10:03:11.754 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:03:11.774 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:03:11.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:03:11.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:11.896 INFO:teuthology.orchestra.run.smithi045.stdout:client24754 2024-09-10T10:03:11.897 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:03:11.900 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.645+0000 7fc6db0c9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:03:11.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:03:11.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:11.982 INFO:teuthology.orchestra.run.smithi045.stdout:client24754 2024-09-10T10:03:11.983 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968.client24754/status') 2024-09-10T10:03:12.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:03:12.096 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.24754 (3)172.21.15.45:0/3513406857 2024-09-10T10:03:12.096 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-09-10T10:03:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-09-10T10:03:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-09-10T10:03:12.096 INFO:teuthology.orchestra.run.smithi045.stderr:71 bytes copied, 2.5186e-05 s, 2.8 MB/s 2024-09-10T10:03:12.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:03:12.103 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T10:03:12.114 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T10:03:12.192 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T10:03:12.192 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:03:12.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-10T10:03:12.216 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.936+0000 7fc6429db640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.216 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.936+0000 7fc6429db640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.216 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.936+0000 7fc6429db640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.216 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.936+0000 7fc6429db640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.936+0000 7fc6429db640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.939+0000 7fc6451e0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.939+0000 7fc6451e0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.939+0000 7fc6451e0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.939+0000 7fc6451e0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:11.939+0000 7fc6451e0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 -- 172.21.15.45:0/65619944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398102620 msgr2=0x7f8398104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 --2- 172.21.15.45:0/65619944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398102620 0x7f8398104a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8388009860 tx=0x7f838802a340 comp rx=0 tx=0).stop 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 -- 172.21.15.45:0/65619944 shutdown_connections 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 --2- 172.21.15.45:0/65619944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 0x7f8398109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 --2- 172.21.15.45:0/65619944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8398104f50 0x7f8398107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 --2- 172.21.15.45:0/65619944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398102620 0x7f8398104a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 -- 172.21.15.45:0/65619944 >> 172.21.15.45:0/65619944 conn(0x7f83980fc1e0 msgr2=0x7f83980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 -- 172.21.15.45:0/65619944 shutdown_connections 2024-09-10T10:03:12.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.312+0000 7f839d24c640 1 -- 172.21.15.45:0/65619944 wait complete. 2024-09-10T10:03:12.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 Processor -- start 2024-09-10T10:03:12.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 -- start start 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8398102620 0x7f8398197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398104f50 0x7f8398198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 0x7f839819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f839810d810 con 0x7f8398102620 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f839810d690 con 0x7f8398104f50 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.313+0000 7f839d24c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f839810d990 con 0x7f8398107880 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 0x7f839819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 0x7f839819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:46998/0 (socket says 172.21.15.45:46998) 2024-09-10T10:03:12.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 -- 172.21.15.45:0/2053743573 learned_addr learned my addr 172.21.15.45:0/2053743573 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f8396d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8398102620 0x7f8398197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839659c640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398104f50 0x7f8398198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 -- 172.21.15.45:0/2053743573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398104f50 msgr2=0x7f8398198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398104f50 0x7f8398198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 -- 172.21.15.45:0/2053743573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8398102620 msgr2=0x7f8398197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8398102620 0x7f8398197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 -- 172.21.15.45:0/2053743573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839819fa60 con 0x7f8398107880 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f8396d9d640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8398102620 0x7f8398197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839759e640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 0x7f839819f3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f839000eb40 tx=0x7f839000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f837ffff640 1 -- 172.21.15.45:0/2053743573 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390004b20 con 0x7f8398107880 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8398100470 con 0x7f8398107880 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f837ffff640 1 -- 172.21.15.45:0/2053743573 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8390004cc0 con 0x7f8398107880 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.314+0000 7f837ffff640 1 -- 172.21.15.45:0/2053743573 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390010850 con 0x7f8398107880 2024-09-10T10:03:12.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.315+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83981009c0 con 0x7f8398107880 2024-09-10T10:03:12.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.315+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83980fdcd0 con 0x7f8398107880 2024-09-10T10:03:12.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.317+0000 7f837ffff640 1 -- 172.21.15.45:0/2053743573 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f8390005020 con 0x7f8398107880 2024-09-10T10:03:12.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.318+0000 7f837ffff640 1 -- 172.21.15.45:0/2053743573 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f83900cf510 con 0x7f8398107880 2024-09-10T10:03:12.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.321+0000 7f837ffff640 1 -- 172.21.15.45:0/2053743573 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f839009c760 con 0x7f8398107880 2024-09-10T10:03:12.369 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-10T10:03:12.369 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: Standby manager daemon y restarted 2024-09-10T10:03:12.369 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: Standby manager daemon y started 2024-09-10T10:03:12.370 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:03:12.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.500+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f839802d7c0 con 0x7f8398107880 2024-09-10T10:03:12.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.508+0000 7f837ffff640 1 -- 172.21.15.45:0/2053743573 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v26)=0 v26) ==== 129+0+0 (secure 0 0 0) 0x7f839009c060 con 0x7f8398107880 2024-09-10T10:03:12.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 msgr2=0x7f839819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 0x7f839819f3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f839000eb40 tx=0x7f839000c2d0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 shutdown_connections 2024-09-10T10:03:12.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8398107880 0x7f839819f3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8398104f50 0x7f8398198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 --2- 172.21.15.45:0/2053743573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8398102620 0x7f8398197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 >> 172.21.15.45:0/2053743573 conn(0x7f83980fc1e0 msgr2=0x7f8398102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:12.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 shutdown_connections 2024-09-10T10:03:12.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.511+0000 7f839d24c640 1 -- 172.21.15.45:0/2053743573 wait complete. 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: Standby manager daemon y restarted 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: Standby manager daemon y started 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: Active manager daemon x restarted 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: Activating manager daemon x 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: mgrmap e20: x(active, starting, since 0.0120853s), standbys: z, y 2024-09-10T10:03:12.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:03:12.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:03:12.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:03:12.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:03:12.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:03:12.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:03:12.526 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:03:12.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-10T10:03:12.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:03:12.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:03:12.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:03:12.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:03:12.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:03:12.527 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: Manager daemon x is now available 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:03:12.528 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:12 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:03:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:03:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:03:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:03:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: Active manager daemon x restarted 2024-09-10T10:03:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: Activating manager daemon x 2024-09-10T10:03:12.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: mgrmap e20: x(active, starting, since 0.0120853s), standbys: z, y 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:03:12.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:03:12.652 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:03:12.652 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:03:12.652 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:03:12.652 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:03:12.652 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:03:12.652 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:03:12.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: Manager daemon x is now available 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:03:12.654 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:12 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:03:12.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-09-10T10:03:12.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: Standby manager daemon y restarted 2024-09-10T10:03:12.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: Standby manager daemon y started 2024-09-10T10:03:12.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T10:03:12.662 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.? 172.21.15.47:0/2009933361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: Active manager daemon x restarted 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: Activating manager daemon x 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: mgrmap e20: x(active, starting, since 0.0120853s), standbys: z, y 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T10:03:12.663 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T10:03:12.664 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T10:03:12.665 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T10:03:12.666 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: Manager daemon x is now available 2024-09-10T10:03:12.666 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:12.666 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T10:03:12.666 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:12 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 -- 172.21.15.45:0/1259371720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8103710 msgr2=0x7fb8a8109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/1259371720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8103710 0x7fb8a8109fa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009ca0 tx=0x7fb8a002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 -- 172.21.15.45:0/1259371720 shutdown_connections 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/1259371720 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8103710 0x7fb8a8109fa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/1259371720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a8102d50 0x7fb8a81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/1259371720 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8a8101b60 0x7fb8a8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 -- 172.21.15.45:0/1259371720 >> 172.21.15.45:0/1259371720 conn(0x7fb8a80fd2e0 msgr2=0x7fb8a80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:12.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 -- 172.21.15.45:0/1259371720 shutdown_connections 2024-09-10T10:03:12.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.688+0000 7fb8aea5e640 1 -- 172.21.15.45:0/1259371720 wait complete. 2024-09-10T10:03:12.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.689+0000 7fb8aea5e640 1 Processor -- start 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.689+0000 7fb8aea5e640 1 -- start start 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.689+0000 7fb8aea5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8101b60 0x7fb8a8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.689+0000 7fb8aea5e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a8102d50 0x7fb8a81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8aea5e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8a8103710 0x7fb8a819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8aea5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a810d7c0 con 0x7fb8a8101b60 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8aea5e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a810d640 con 0x7fb8a8102d50 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8aea5e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a810d940 con 0x7fb8a8103710 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8101b60 0x7fb8a8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8101b60 0x7fb8a8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49466/0 (socket says 172.21.15.45:49466) 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8a7fff640 1 -- 172.21.15.45:0/3696434631 learned_addr learned my addr 172.21.15.45:0/3696434631 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8a77fe640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a8102d50 0x7fb8a81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8acffb640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8a8103710 0x7fb8a819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8acffb640 1 -- 172.21.15.45:0/3696434631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a8102d50 msgr2=0x7fb8a81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8acffb640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a8102d50 0x7fb8a81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8acffb640 1 -- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8101b60 msgr2=0x7fb8a8197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8acffb640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8101b60 0x7fb8a8197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8acffb640 1 -- 172.21.15.45:0/3696434631 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a819f9b0 con 0x7fb8a8103710 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8a7fff640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8101b60 0x7fb8a8197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.690+0000 7fb8acffb640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8a8103710 0x7fb8a819f310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0031ae0 tx=0x7fb8a0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.691+0000 7fb8a57fa640 1 -- 172.21.15.45:0/3696434631 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0003b70 con 0x7fb8a8103710 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.691+0000 7fb8a57fa640 1 -- 172.21.15.45:0/3696434631 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a0003d10 con 0x7fb8a8103710 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.691+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a819fc40 con 0x7fb8a8103710 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.691+0000 7fb8a57fa640 1 -- 172.21.15.45:0/3696434631 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a002fc40 con 0x7fb8a8103710 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.691+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a81a01b0 con 0x7fb8a8103710 2024-09-10T10:03:12.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.692+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a8108f00 con 0x7fb8a8103710 2024-09-10T10:03:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.694+0000 7fb8a57fa640 1 -- 172.21.15.45:0/3696434631 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fb8a0070a70 con 0x7fb8a8103710 2024-09-10T10:03:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.694+0000 7fb8a57fa640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8900a2ac0 0x7fb8900a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.694+0000 7fb8a57fa640 1 -- 172.21.15.45:0/3696434631 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb8a00efee0 con 0x7fb8a8103710 2024-09-10T10:03:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.694+0000 7fb8a7fff640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8900a2ac0 0x7fb8900a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:12.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.695+0000 7fb8a7fff640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8900a2ac0 0x7fb8900a4f80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8199270 tx=0x7fb898005f90 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:12.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.697+0000 7fb8a57fa640 1 -- 172.21.15.45:0/3696434631 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8a00bd020 con 0x7fb8a8103710 2024-09-10T10:03:12.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.877+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb8a8062df0 con 0x7fb8a8103710 2024-09-10T10:03:12.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.883+0000 7fb8a57fa640 1 -- 172.21.15.45:0/3696434631 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7fb8a00bc920 con 0x7fb8a8103710 2024-09-10T10:03:12.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8900a2ac0 msgr2=0x7fb8900a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8900a2ac0 0x7fb8900a4f80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8199270 tx=0x7fb898005f90 comp rx=0 tx=0).stop 2024-09-10T10:03:12.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8a8103710 msgr2=0x7fb8a819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:12.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8a8103710 0x7fb8a819f310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0031ae0 tx=0x7fb8a0004ab0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 shutdown_connections 2024-09-10T10:03:12.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb8a8103710 0x7fb8a819f310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8900a2ac0 0x7fb8900a4f80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb8a8102d50 0x7fb8a81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 --2- 172.21.15.45:0/3696434631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8a8101b60 0x7fb8a8197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:12.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 >> 172.21.15.45:0/3696434631 conn(0x7fb8a80fd2e0 msgr2=0x7fb8a81060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:12.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 shutdown_connections 2024-09-10T10:03:12.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:12.886+0000 7fb8aea5e640 1 -- 172.21.15.45:0/3696434631 wait complete. 2024-09-10T10:03:12.900 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-10T10:03:13.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.008+0000 7fbb5a39f640 1 -- 172.21.15.45:0/293240365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb54105340 msgr2=0x7fbb54110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.008+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/293240365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb54105340 0x7fbb54110490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009ca0 tx=0x7fbb4c02a8b0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 -- 172.21.15.45:0/293240365 shutdown_connections 2024-09-10T10:03:13.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/293240365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb54105340 0x7fbb54110490 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/293240365 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb54074fc0 0x7fbb5410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/293240365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb54074680 0x7fbb54074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 -- 172.21.15.45:0/293240365 >> 172.21.15.45:0/293240365 conn(0x7fbb540fd0d0 msgr2=0x7fbb540ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:13.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 -- 172.21.15.45:0/293240365 shutdown_connections 2024-09-10T10:03:13.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 -- 172.21.15.45:0/293240365 wait complete. 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.009+0000 7fbb5a39f640 1 Processor -- start 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5a39f640 1 -- start start 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5a39f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb54074680 0x7fbb5419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5a39f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb54074fc0 0x7fbb5419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5a39f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb541a3920 0x7fbb541a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5a39f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54113d40 con 0x7fbb541a3920 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5a39f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54113bc0 con 0x7fbb54074680 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5a39f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54113ec0 con 0x7fbb54074fc0 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb541a3920 0x7fbb541a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb541a3920 0x7fbb541a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49488/0 (socket says 172.21.15.45:49488) 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 -- 172.21.15.45:0/721407626 learned_addr learned my addr 172.21.15.45:0/721407626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:13.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb58b9c640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb54074fc0 0x7fbb5419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb5939d640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb54074680 0x7fbb5419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 -- 172.21.15.45:0/721407626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb54074fc0 msgr2=0x7fbb5419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb54074fc0 0x7fbb5419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 -- 172.21.15.45:0/721407626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb54074680 msgr2=0x7fbb5419e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb54074680 0x7fbb5419e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 -- 172.21.15.45:0/721407626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb541a6380 con 0x7fbb541a3920 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.010+0000 7fbb59b9e640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb541a3920 0x7fbb541a5d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009dc0 tx=0x7fbb4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.011+0000 7fbb4a7fc640 1 -- 172.21.15.45:0/721407626 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c004450 con 0x7fbb541a3920 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.011+0000 7fbb4a7fc640 1 -- 172.21.15.45:0/721407626 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c040070 con 0x7fbb541a3920 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.011+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb541a6610 con 0x7fbb541a3920 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.011+0000 7fbb4a7fc640 1 -- 172.21.15.45:0/721407626 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c045e20 con 0x7fbb541a3920 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.011+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb541a6b60 con 0x7fbb541a3920 2024-09-10T10:03:13.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.012+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb54074a80 con 0x7fbb541a3920 2024-09-10T10:03:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.013+0000 7fbb4a7fc640 1 -- 172.21.15.45:0/721407626 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fbb4c0045f0 con 0x7fbb541a3920 2024-09-10T10:03:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.014+0000 7fbb4a7fc640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fbb400a2970 0x7fbb400a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.014+0000 7fbb5939d640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fbb400a2970 0x7fbb400a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.014+0000 7fbb4a7fc640 1 -- 172.21.15.45:0/721407626 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fbb4c0f0200 con 0x7fbb541a3920 2024-09-10T10:03:13.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.014+0000 7fbb5939d640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fbb400a2970 0x7fbb400a4e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb50005d00 tx=0x7fbb50005c50 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.018+0000 7fbb4a7fc640 1 -- 172.21.15.45:0/721407626 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb4c0bd320 con 0x7fbb541a3920 2024-09-10T10:03:13.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.197+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fbb5402d860 con 0x7fbb541a3920 2024-09-10T10:03:13.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.201+0000 7fbb4a7fc640 1 -- 172.21.15.45:0/721407626 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v28)=0 v28) ==== 133+0+0 (secure 0 0 0) 0x7fbb4c0bcc20 con 0x7fbb541a3920 2024-09-10T10:03:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fbb400a2970 msgr2=0x7fbb400a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fbb400a2970 0x7fbb400a4e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb50005d00 tx=0x7fbb50005c50 comp rx=0 tx=0).stop 2024-09-10T10:03:13.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb541a3920 msgr2=0x7fbb541a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb541a3920 0x7fbb541a5d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009dc0 tx=0x7fbb4c003c20 comp rx=0 tx=0).stop 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 shutdown_connections 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb541a3920 0x7fbb541a5d10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbb54074fc0 0x7fbb5419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fbb400a2970 0x7fbb400a4e30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 --2- 172.21.15.45:0/721407626 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb54074680 0x7fbb5419e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 >> 172.21.15.45:0/721407626 conn(0x7fbb540fd0d0 msgr2=0x7fbb54110030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 shutdown_connections 2024-09-10T10:03:13.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.203+0000 7fbb5a39f640 1 -- 172.21.15.45:0/721407626 wait complete. 2024-09-10T10:03:13.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 -- 172.21.15.45:0/2545015185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144100a40 msgr2=0x7f7144100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 --2- 172.21.15.45:0/2545015185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144100a40 0x7f7144100e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7134009860 tx=0x7f713402a340 comp rx=0 tx=0).stop 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 -- 172.21.15.45:0/2545015185 shutdown_connections 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 --2- 172.21.15.45:0/2545015185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71441025f0 0x7f7144108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 --2- 172.21.15.45:0/2545015185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7144101c30 0x7f71441020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 --2- 172.21.15.45:0/2545015185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144100a40 0x7f7144100e40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 -- 172.21.15.45:0/2545015185 >> 172.21.15.45:0/2545015185 conn(0x7f71440fc1e0 msgr2=0x7f71440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 -- 172.21.15.45:0/2545015185 shutdown_connections 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.323+0000 7f7149766640 1 -- 172.21.15.45:0/2545015185 wait complete. 2024-09-10T10:03:13.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 Processor -- start 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 -- start start 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7144100a40 0x7f714410b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144101c30 0x7f7144109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71441025f0 0x7f7144109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f714410d810 con 0x7f7144101c30 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f714410d690 con 0x7f71441025f0 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7149766640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f714410d990 con 0x7f7144100a40 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f71427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144101c30 0x7f7144109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f71427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144101c30 0x7f7144109a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54386/0 (socket says 172.21.15.45:54386) 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f71427fc640 1 -- 172.21.15.45:0/3406684596 learned_addr learned my addr 172.21.15.45:0/3406684596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f7142ffd640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7144100a40 0x7f714410b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.324+0000 7f71437fe640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71441025f0 0x7f7144109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7142ffd640 1 -- 172.21.15.45:0/3406684596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71441025f0 msgr2=0x7f7144109f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7142ffd640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71441025f0 0x7f7144109f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7142ffd640 1 -- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144101c30 msgr2=0x7f7144109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7142ffd640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144101c30 0x7f7144109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7142ffd640 1 -- 172.21.15.45:0/3406684596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714410a7d0 con 0x7f7144100a40 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f71427fc640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144101c30 0x7f7144109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f71437fe640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71441025f0 0x7f7144109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7142ffd640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7144100a40 0x7f714410b3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7134035040 tx=0x7f7134033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f712bfff640 1 -- 172.21.15.45:0/3406684596 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134037040 con 0x7f7144100a40 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f712bfff640 1 -- 172.21.15.45:0/3406684596 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7134004040 con 0x7f7144100a40 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71441a1ec0 con 0x7f7144100a40 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f712bfff640 1 -- 172.21.15.45:0/3406684596 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134005650 con 0x7f7144100a40 2024-09-10T10:03:13.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.325+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71441a22d0 con 0x7f7144100a40 2024-09-10T10:03:13.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.326+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7144107e80 con 0x7f7144100a40 2024-09-10T10:03:13.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.327+0000 7f712bfff640 1 -- 172.21.15.45:0/3406684596 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f7134042020 con 0x7f7144100a40 2024-09-10T10:03:13.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.328+0000 7f712bfff640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f712c0a2a40 0x7f712c0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.328+0000 7f71427fc640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f712c0a2a40 0x7f712c0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.328+0000 7f712bfff640 1 -- 172.21.15.45:0/3406684596 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f71340ee4f0 con 0x7f7144100a40 2024-09-10T10:03:13.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.329+0000 7f71427fc640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f712c0a2a40 0x7f712c0a4f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71380096c0 tx=0x7f7138009260 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.334+0000 7f712bfff640 1 -- 172.21.15.45:0/3406684596 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7134049020 con 0x7f7144100a40 2024-09-10T10:03:13.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.546+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 --> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7144062630 con 0x7f712c0a2a40 2024-09-10T10:03:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.612+0000 7f712bfff640 1 -- 172.21.15.45:0/3406684596 <== mgr.15060 v2:172.21.15.45:6800/506901738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7144062630 con 0x7f712c0a2a40 2024-09-10T10:03:13.613 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path / 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.613+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f712c0a2a40 msgr2=0x7f712c0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.613+0000 7f7149766640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f712c0a2a40 0x7f712c0a4f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71380096c0 tx=0x7f7138009260 comp rx=0 tx=0).stop 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.613+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7144100a40 msgr2=0x7f714410b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.613+0000 7f7149766640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7144100a40 0x7f714410b3c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7134035040 tx=0x7f7134033d90 comp rx=0 tx=0).stop 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 shutdown_connections 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f712c0a2a40 0x7f712c0a4f00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71441025f0 0x7f7144109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7144101c30 0x7f7144109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 --2- 172.21.15.45:0/3406684596 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7144100a40 0x7f714410b3c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 >> 172.21.15.45:0/3406684596 conn(0x7f71440fc1e0 msgr2=0x7f7144105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:13.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 shutdown_connections 2024-09-10T10:03:13.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.614+0000 7f7149766640 1 -- 172.21.15.45:0/3406684596 wait complete. 2024-09-10T10:03:13.626 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-10T10:03:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 -- 172.21.15.45:0/2198070654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c101b50 msgr2=0x7fb26c101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 --2- 172.21.15.45:0/2198070654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c101b50 0x7fb26c101f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009900 tx=0x7fb25c02a410 comp rx=0 tx=0).stop 2024-09-10T10:03:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 -- 172.21.15.45:0/2198070654 shutdown_connections 2024-09-10T10:03:13.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 --2- 172.21.15.45:0/2198070654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c103700 0x7fb26c109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 --2- 172.21.15.45:0/2198070654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb26c102d40 0x7fb26c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 --2- 172.21.15.45:0/2198070654 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c101b50 0x7fb26c101f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 -- 172.21.15.45:0/2198070654 >> 172.21.15.45:0/2198070654 conn(0x7fb26c0fd2b0 msgr2=0x7fb26c0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 -- 172.21.15.45:0/2198070654 shutdown_connections 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.749+0000 7fb27131a640 1 -- 172.21.15.45:0/2198070654 wait complete. 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.750+0000 7fb27131a640 1 Processor -- start 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.750+0000 7fb27131a640 1 -- start start 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.750+0000 7fb27131a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c101b50 0x7fb26c197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb27131a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb26c102d40 0x7fb26c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb27131a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c103700 0x7fb26c19f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb27131a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10d820 con 0x7fb26c101b50 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb27131a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10d6a0 con 0x7fb26c102d40 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb27131a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10d9a0 con 0x7fb26c103700 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c101b50 0x7fb26c197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c101b50 0x7fb26c197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54410/0 (socket says 172.21.15.45:54410) 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb26c102d40 0x7fb26c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 -- 172.21.15.45:0/2506251251 learned_addr learned my addr 172.21.15.45:0/2506251251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26b7fe640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c103700 0x7fb26c19f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 -- 172.21.15.45:0/2506251251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c103700 msgr2=0x7fb26c19f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c103700 0x7fb26c19f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 -- 172.21.15.45:0/2506251251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb26c102d40 msgr2=0x7fb26c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb26c102d40 0x7fb26c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:13.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 -- 172.21.15.45:0/2506251251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c19fa10 con 0x7fb26c101b50 2024-09-10T10:03:13.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26a7fc640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb26c102d40 0x7fb26c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:13.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26b7fe640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c103700 0x7fb26c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:13.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.751+0000 7fb26affd640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c101b50 0x7fb26c197df0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb25c031440 tx=0x7fb25c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.752+0000 7fb253fff640 1 -- 172.21.15.45:0/2506251251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c03f030 con 0x7fb26c101b50 2024-09-10T10:03:13.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.752+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c19fca0 con 0x7fb26c101b50 2024-09-10T10:03:13.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.752+0000 7fb253fff640 1 -- 172.21.15.45:0/2506251251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25c003ce0 con 0x7fb26c101b50 2024-09-10T10:03:13.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.753+0000 7fb253fff640 1 -- 172.21.15.45:0/2506251251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c035070 con 0x7fb26c101b50 2024-09-10T10:03:13.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.753+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1a0190 con 0x7fb26c101b50 2024-09-10T10:03:13.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.754+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c108e70 con 0x7fb26c101b50 2024-09-10T10:03:13.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.755+0000 7fb253fff640 1 -- 172.21.15.45:0/2506251251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fb25c042030 con 0x7fb26c101b50 2024-09-10T10:03:13.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.756+0000 7fb253fff640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb2540a2a40 0x7fb2540a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.756+0000 7fb26a7fc640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb2540a2a40 0x7fb2540a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.756+0000 7fb253fff640 1 -- 172.21.15.45:0/2506251251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb25c0ef510 con 0x7fb26c101b50 2024-09-10T10:03:13.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.757+0000 7fb26a7fc640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb2540a2a40 0x7fb2540a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb26c199300 tx=0x7fb260005f90 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:13.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.762+0000 7fb253fff640 1 -- 172.21.15.45:0/2506251251 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb25c0bc650 con 0x7fb26c101b50 2024-09-10T10:03:13.867 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-mon[31114]: from='client.? ' entity='client.admin' 2024-09-10T10:03:13.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-mon[31114]: mgrmap e21: x(active, since 1.01893s), standbys: z, y 2024-09-10T10:03:13.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-mon[31114]: from='client.? ' entity='client.admin' 2024-09-10T10:03:13.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/721407626' entity='client.admin' 2024-09-10T10:03:13.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:13 smithi162 ceph-mon[34963]: from='client.? ' entity='client.admin' 2024-09-10T10:03:13.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:13 smithi162 ceph-mon[34963]: mgrmap e21: x(active, since 1.01893s), standbys: z, y 2024-09-10T10:03:13.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:13 smithi162 ceph-mon[34963]: from='client.? ' entity='client.admin' 2024-09-10T10:03:13.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:13 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/721407626' entity='client.admin' 2024-09-10T10:03:13.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:13 smithi047 ceph-mon[35244]: from='client.? ' entity='client.admin' 2024-09-10T10:03:13.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:13 smithi047 ceph-mon[35244]: mgrmap e21: x(active, since 1.01893s), standbys: z, y 2024-09-10T10:03:13.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:13 smithi047 ceph-mon[35244]: from='client.? ' entity='client.admin' 2024-09-10T10:03:13.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:13 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/721407626' entity='client.admin' 2024-09-10T10:03:13.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:13.989+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 --> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fb26c0626d0 con 0x7fb2540a2a40 2024-09-10T10:03:14.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.045+0000 7fb253fff640 1 -- 172.21.15.45:0/2506251251 <== mgr.15060 v2:172.21.15.45:6800/506901738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fb26c0626d0 con 0x7fb2540a2a40 2024-09-10T10:03:14.046 INFO:teuthology.orchestra.run.smithi045.stdout:Retention added to path / 2024-09-10T10:03:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb2540a2a40 msgr2=0x7fb2540a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb2540a2a40 0x7fb2540a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb26c199300 tx=0x7fb260005f90 comp rx=0 tx=0).stop 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c101b50 msgr2=0x7fb26c197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c101b50 0x7fb26c197df0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb25c031440 tx=0x7fb25c032ce0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 shutdown_connections 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb26c103700 0x7fb26c19f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb2540a2a40 0x7fb2540a4f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb26c102d40 0x7fb26c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 --2- 172.21.15.45:0/2506251251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb26c101b50 0x7fb26c197df0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 >> 172.21.15.45:0/2506251251 conn(0x7fb26c0fd2b0 msgr2=0x7fb26c106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 shutdown_connections 2024-09-10T10:03:14.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.047+0000 7fb27131a640 1 -- 172.21.15.45:0/2506251251 wait complete. 2024-09-10T10:03:14.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T10:03:14.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:13.865+0000 7fc6288a8640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:10:03:13] ENGINE Error in HTTPServer.serve 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: Traceback (most recent call last): 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._connections.run(self.expiration_interval) 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._run(expiration_interval) 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:03:14.151 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: s = self.context.wrap_socket( 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: return self.sslsocket_class._create( 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self.do_handshake() 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: self._sslobj.do_handshake() 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:03:14.152 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:03:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 -- 172.21.15.45:0/3296536077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd0101670 msgr2=0x7efcd0108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 --2- 172.21.15.45:0/3296536077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd0101670 0x7efcd0108f10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009ca0 tx=0x7efcc802a8b0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 -- 172.21.15.45:0/3296536077 shutdown_connections 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 --2- 172.21.15.45:0/3296536077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd0101670 0x7efcd0108f10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 --2- 172.21.15.45:0/3296536077 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd0100cb0 0x7efcd0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 --2- 172.21.15.45:0/3296536077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd01020f0 0x7efcd01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 -- 172.21.15.45:0/3296536077 >> 172.21.15.45:0/3296536077 conn(0x7efcd00fb970 msgr2=0x7efcd00fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 -- 172.21.15.45:0/3296536077 shutdown_connections 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.165+0000 7efcd56b7640 1 -- 172.21.15.45:0/3296536077 wait complete. 2024-09-10T10:03:14.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 Processor -- start 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 -- start start 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd0100cb0 0x7efcd0197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd0101670 0x7efcd01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd01020f0 0x7efcd019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010c770 con 0x7efcd01020f0 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010c5f0 con 0x7efcd0100cb0 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcd56b7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efcd010c8f0 con 0x7efcd0101670 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd01020f0 0x7efcd019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcce7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd0101670 0x7efcd01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efccf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd01020f0 0x7efcd019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54428/0 (socket says 172.21.15.45:54428) 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efcceffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd0100cb0 0x7efcd0197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.166+0000 7efccf7fe640 1 -- 172.21.15.45:0/547728661 learned_addr learned my addr 172.21.15.45:0/547728661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:14.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcce7fc640 1 -- 172.21.15.45:0/547728661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd0100cb0 msgr2=0x7efcd0197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcce7fc640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd0100cb0 0x7efcd0197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcce7fc640 1 -- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd01020f0 msgr2=0x7efcd019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcce7fc640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd01020f0 0x7efcd019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcce7fc640 1 -- 172.21.15.45:0/547728661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd019fab0 con 0x7efcd0101670 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcce7fc640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd0101670 0x7efcd01983d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e9c0 tx=0x7efcc400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcb7fff640 1 -- 172.21.15.45:0/547728661 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc400ccc0 con 0x7efcd0101670 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd019fda0 con 0x7efcd0101670 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcb7fff640 1 -- 172.21.15.45:0/547728661 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc40035f0 con 0x7efcd0101670 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcb7fff640 1 -- 172.21.15.45:0/547728661 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc400ce60 con 0x7efcd0101670 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.167+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd01a0260 con 0x7efcd0101670 2024-09-10T10:03:14.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.169+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd01a04f0 con 0x7efcd0101670 2024-09-10T10:03:14.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.171+0000 7efcb7fff640 1 -- 172.21.15.45:0/547728661 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7efcc40037a0 con 0x7efcd0101670 2024-09-10T10:03:14.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.171+0000 7efcb7fff640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efcb80a2a40 0x7efcb80a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.171+0000 7efcb7fff640 1 -- 172.21.15.45:0/547728661 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7efcc40cf250 con 0x7efcd0101670 2024-09-10T10:03:14.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.171+0000 7efcceffd640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efcb80a2a40 0x7efcb80a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.172+0000 7efcceffd640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efcb80a2a40 0x7efcb80a4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009bd0 tx=0x7efcc0005f90 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.180+0000 7efcb7fff640 1 -- 172.21.15.45:0/547728661 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcc409c390 con 0x7efcd0101670 2024-09-10T10:03:14.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.381+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 --> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efcd01a06c0 con 0x7efcb80a2a40 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.388+0000 7efcb7fff640 1 -- 172.21.15.45:0/547728661 <== mgr.15060 v2:172.21.15.45:6800/506901738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7efcd01a06c0 con 0x7efcb80a2a40 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T10:03:13", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efcb80a2a40 msgr2=0x7efcb80a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efcb80a2a40 0x7efcb80a4f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009bd0 tx=0x7efcc0005f90 comp rx=0 tx=0).stop 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd0101670 msgr2=0x7efcd01983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd0101670 0x7efcd01983d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efcc400e9c0 tx=0x7efcc400ee80 comp rx=0 tx=0).stop 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 shutdown_connections 2024-09-10T10:03:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd01020f0 0x7efcd019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efcd0101670 0x7efcd01983d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efcb80a2a40 0x7efcb80a4f00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 --2- 172.21.15.45:0/547728661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efcd0100cb0 0x7efcd0197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 >> 172.21.15.45:0/547728661 conn(0x7efcd00fb970 msgr2=0x7efcd00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 shutdown_connections 2024-09-10T10:03:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.390+0000 7efcd56b7640 1 -- 172.21.15.45:0/547728661 wait complete. 2024-09-10T10:03:14.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T10:03:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3719405167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e640761d0 msgr2=0x7f3e64074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3719405167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e640761d0 0x7f3e64074620 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009900 tx=0x7f3e5402a480 comp rx=0 tx=0).stop 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3719405167 shutdown_connections 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3719405167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e641099d0 0x7f3e6410be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3719405167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e64074bf0 0x7f3e64075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3719405167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e640761d0 0x7f3e64074620 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3719405167 >> 172.21.15.45:0/3719405167 conn(0x7f3e640fd2e0 msgr2=0x7f3e640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3719405167 shutdown_connections 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.511+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3719405167 wait complete. 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.512+0000 7f3e6a9ff640 1 Processor -- start 2024-09-10T10:03:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.512+0000 7f3e6a9ff640 1 -- start start 2024-09-10T10:03:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e6a9ff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e64074bf0 0x7f3e64199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e6a9ff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e640761d0 0x7f3e6419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e6a9ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e641099d0 0x7f3e641a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e6a9ff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410f9d0 con 0x7f3e641099d0 2024-09-10T10:03:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e6a9ff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410f850 con 0x7f3e64074bf0 2024-09-10T10:03:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e6a9ff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6410fb50 con 0x7f3e640761d0 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e641099d0 0x7f3e641a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e641099d0 0x7f3e641a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54448/0 (socket says 172.21.15.45:54448) 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 -- 172.21.15.45:0/3259112467 learned_addr learned my addr 172.21.15.45:0/3259112467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 -- 172.21.15.45:0/3259112467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e640761d0 msgr2=0x7f3e6419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e640761d0 0x7f3e6419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 -- 172.21.15.45:0/3259112467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e64074bf0 msgr2=0x7f3e64199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e64074bf0 0x7f3e64199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.513+0000 7f3e68f9c640 1 -- 172.21.15.45:0/3259112467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e641a1c10 con 0x7f3e641099d0 2024-09-10T10:03:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.514+0000 7f3e68f9c640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e641099d0 0x7f3e641a15a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00eb10 tx=0x7f3e5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.522+0000 7f3e617fa640 1 -- 172.21.15.45:0/3259112467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c004b20 con 0x7f3e641099d0 2024-09-10T10:03:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.522+0000 7f3e617fa640 1 -- 172.21.15.45:0/3259112467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5c004cc0 con 0x7f3e641099d0 2024-09-10T10:03:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.523+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e641a1f00 con 0x7f3e641099d0 2024-09-10T10:03:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.523+0000 7f3e617fa640 1 -- 172.21.15.45:0/3259112467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c010760 con 0x7f3e641099d0 2024-09-10T10:03:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.523+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e641a2470 con 0x7f3e641099d0 2024-09-10T10:03:14.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.524+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e64108790 con 0x7f3e641099d0 2024-09-10T10:03:14.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.525+0000 7f3e617fa640 1 -- 172.21.15.45:0/3259112467 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f3e5c005020 con 0x7f3e641099d0 2024-09-10T10:03:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.526+0000 7f3e617fa640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f3e4c0a2e20 0x7f3e4c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.526+0000 7f3e63fff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f3e4c0a2e20 0x7f3e4c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.526+0000 7f3e617fa640 1 -- 172.21.15.45:0/3259112467 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f3e5c0cfbd0 con 0x7f3e641099d0 2024-09-10T10:03:14.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.526+0000 7f3e63fff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f3e4c0a2e20 0x7f3e4c0a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e540096c0 tx=0x7f3e54031400 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.531+0000 7f3e617fa640 1 -- 172.21.15.45:0/3259112467 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e5c09c800 con 0x7f3e641099d0 2024-09-10T10:03:14.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.770+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 --> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3e64062630 con 0x7f3e4c0a2e20 2024-09-10T10:03:14.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.776+0000 7f3e617fa640 1 -- 172.21.15.45:0/3259112467 <== mgr.15060 v2:172.21.15.45:6800/506901738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3e64062630 con 0x7f3e4c0a2e20 2024-09-10T10:03:14.777 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f3e4c0a2e20 msgr2=0x7f3e4c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f3e4c0a2e20 0x7f3e4c0a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e540096c0 tx=0x7f3e54031400 comp rx=0 tx=0).stop 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e641099d0 msgr2=0x7f3e641a15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e641099d0 0x7f3e641a15a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00eb10 tx=0x7f3e5c00c2d0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 shutdown_connections 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3e641099d0 0x7f3e641a15a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3e640761d0 0x7f3e6419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f3e4c0a2e20 0x7f3e4c0a52e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 --2- 172.21.15.45:0/3259112467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3e64074bf0 0x7f3e64199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 >> 172.21.15.45:0/3259112467 conn(0x7f3e640fd2e0 msgr2=0x7f3e64105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 shutdown_connections 2024-09-10T10:03:14.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:03:14.778+0000 7f3e6a9ff640 1 -- 172.21.15.45:0/3259112467 wait complete. 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.795 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:14 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-10T10:03:14.824 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-10T10:03:14.834 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-10T10:03:14.933 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:03:14.934 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-10T10:03:14.935 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:14.935 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:14.935 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.935 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.935 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.935 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:14 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.936 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.937 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:14 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:14.938 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T10:03:14.993 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:03:14.994 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T10:03:14.994 INFO:tasks.workunit:timeout=3h 2024-09-10T10:03:14.994 INFO:tasks.workunit:cleanup=True 2024-09-10T10:03:14.994 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-09-10 10:01:50.692403479 +0000 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-09-10 10:01:50.692403479 +0000 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-09-10 10:03:11.676606885 +0000 2024-09-10T10:03:15.011 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-09-10 10:01:50.692403479 +0000 2024-09-10T10:03:15.011 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T10:03:15.012 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T10:03:15.081 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:03:15.138 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T10:03:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: [10/Sep/2024:10:03:13] ENGINE Bus STARTING 2024-09-10T10:03:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:15.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: [10/Sep/2024:10:03:13] ENGINE Serving on http://172.21.15.45:8765 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: [10/Sep/2024:10:03:13] ENGINE Serving on https://172.21.15.45:7150 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: [10/Sep/2024:10:03:13] ENGINE Bus STARTED 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: [10/Sep/2024:10:03:13] ENGINE Error in HTTPServer.serve 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout: Traceback (most recent call last): 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout: self._run(expiration_interval) 2024-09-10T10:03:15.901 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: s = self.context.wrap_socket( 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: return self.sslsocket_class._create( 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: self.do_handshake() 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: self._sslobj.do_handshake() 2024-09-10T10:03:15.902 INFO:journalctl@ceph.mon.c.smithi162.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi162 to 176.0M 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Unable to set osd_memory_target on smithi162 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.903 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.904 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.904 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.904 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.904 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.904 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.904 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.905 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-10T10:03:15.905 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.905 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-10T10:03:15.905 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-10T10:03:15.905 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Unable to set osd_memory_target on smithi045 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-10T10:03:15.906 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:15.906 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:15.906 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-10T10:03:15.906 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T10:03:15.906 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:15 smithi162 ceph-mon[34963]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: [10/Sep/2024:10:03:13] ENGINE Bus STARTING 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: [10/Sep/2024:10:03:13] ENGINE Serving on http://172.21.15.45:8765 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: [10/Sep/2024:10:03:13] ENGINE Serving on https://172.21.15.45:7150 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: [10/Sep/2024:10:03:13] ENGINE Bus STARTED 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: [10/Sep/2024:10:03:13] ENGINE Error in HTTPServer.serve 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout: Traceback (most recent call last): 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._run(expiration_interval) 2024-09-10T10:03:15.907 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: s = self.context.wrap_socket( 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: return self.sslsocket_class._create( 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: self.do_handshake() 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:03:15.908 INFO:journalctl@ceph.mon.a.smithi045.stdout: self._sslobj.do_handshake() 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi162 to 176.0M 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Unable to set osd_memory_target on smithi162 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.909 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Unable to set osd_memory_target on smithi045 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-10T10:03:15.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:15 smithi045 ceph-mon[31114]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: [10/Sep/2024:10:03:13] ENGINE Bus STARTING 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: [10/Sep/2024:10:03:13] ENGINE Serving on http://172.21.15.45:8765 2024-09-10T10:03:15.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: [10/Sep/2024:10:03:13] ENGINE Serving on https://172.21.15.45:7150 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: [10/Sep/2024:10:03:13] ENGINE Bus STARTED 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: [10/Sep/2024:10:03:13] ENGINE Error in HTTPServer.serve 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: Traceback (most recent call last): 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: self._connections.run(self.expiration_interval) 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: self._run(expiration_interval) 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: s = self.context.wrap_socket( 2024-09-10T10:03:15.912 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout: return self.sslsocket_class._create( 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout: self.do_handshake() 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout: self._sslobj.do_handshake() 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi162 to 176.0M 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Unable to set osd_memory_target on smithi162 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='client.24820 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.913 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Unable to set osd_memory_target on smithi045 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:15.914 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:15.915 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Updating smithi045:/etc/ceph/ceph.conf 2024-09-10T10:03:15.915 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Updating smithi047:/etc/ceph/ceph.conf 2024-09-10T10:03:15.915 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:15 smithi047 ceph-mon[35244]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T10:03:16.803 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.804 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:16.805 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:16.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: Updating smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:16.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:16.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:16.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:16.911 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:03:17.826 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:17 smithi045 ceph-mon[31114]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:17.826 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:17 smithi045 ceph-mon[31114]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-10T10:03:17.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:17 smithi162 ceph-mon[34963]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:17.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:17 smithi162 ceph-mon[34963]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-10T10:03:17.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:17 smithi047 ceph-mon[35244]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:03:17.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:17 smithi047 ceph-mon[35244]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-10T10:03:19.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:19 smithi162 ceph-mon[34963]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 184 KiB/s wr, 102 op/s 2024-09-10T10:03:19.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:19 smithi045 ceph-mon[31114]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 184 KiB/s wr, 102 op/s 2024-09-10T10:03:19.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:19 smithi047 ceph-mon[35244]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 184 KiB/s wr, 102 op/s 2024-09-10T10:03:21.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:21 smithi162 ceph-mon[34963]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-09-10T10:03:21.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:21 smithi045 ceph-mon[31114]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-09-10T10:03:21.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:21 smithi047 ceph-mon[35244]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-09-10T10:03:23.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:23 smithi162 ceph-mon[34963]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 57 op/s 2024-09-10T10:03:23.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:23 smithi045 ceph-mon[31114]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 57 op/s 2024-09-10T10:03:23.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:23 smithi047 ceph-mon[35244]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 57 op/s 2024-09-10T10:03:25.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:25 smithi162 ceph-mon[34963]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-09-10T10:03:25.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:25 smithi045 ceph-mon[31114]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-09-10T10:03:25.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:25 smithi047 ceph-mon[35244]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 47 op/s 2024-09-10T10:03:26.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:26 smithi162 ceph-mon[34963]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-09-10T10:03:26.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:26 smithi045 ceph-mon[31114]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-09-10T10:03:26.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:26 smithi047 ceph-mon[35244]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 43 op/s 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 -- 172.21.15.45:0/631981518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f0102cc0 msgr2=0x7fb1f01030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/631981518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f0102cc0 0x7fb1f01030a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009900 tx=0x7fb1e002a480 comp rx=0 tx=0).stop 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 -- 172.21.15.45:0/631981518 shutdown_connections 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/631981518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f010c110 0x7fb1f010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/631981518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1f01035e0 0x7fb1f010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/631981518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f0102cc0 0x7fb1f01030a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 -- 172.21.15.45:0/631981518 >> 172.21.15.45:0/631981518 conn(0x7fb1f00fc6b0 msgr2=0x7fb1f00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 -- 172.21.15.45:0/631981518 shutdown_connections 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.084+0000 7fb1f4faf640 1 -- 172.21.15.45:0/631981518 wait complete. 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.085+0000 7fb1f4faf640 1 Processor -- start 2024-09-10T10:03:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.085+0000 7fb1f4faf640 1 -- start start 2024-09-10T10:03:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.085+0000 7fb1f4faf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f0102cc0 0x7fb1f01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.085+0000 7fb1f4faf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1f01035e0 0x7fb1f0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.085+0000 7fb1f4faf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f010c110 0x7fb1f019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.085+0000 7fb1f4faf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0111ef0 con 0x7fb1f010c110 2024-09-10T10:03:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.085+0000 7fb1f4faf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0111d70 con 0x7fb1f0102cc0 2024-09-10T10:03:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1f4faf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112070 con 0x7fb1f01035e0 2024-09-10T10:03:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f010c110 0x7fb1f019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f010c110 0x7fb1f019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47436/0 (socket says 172.21.15.45:47436) 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 -- 172.21.15.45:0/39697414 learned_addr learned my addr 172.21.15.45:0/39697414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1f01035e0 0x7fb1f0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f0102cc0 0x7fb1f01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 -- 172.21.15.45:0/39697414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1f01035e0 msgr2=0x7fb1f0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1f01035e0 0x7fb1f0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 -- 172.21.15.45:0/39697414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f0102cc0 msgr2=0x7fb1f01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f0102cc0 0x7fb1f01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 -- 172.21.15.45:0/39697414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f019d2d0 con 0x7fb1f010c110 2024-09-10T10:03:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1f01035e0 0x7fb1f0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1eed9d640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f010c110 0x7fb1f019cba0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800cce0 tx=0x7fb1e800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1e77fe640 1 -- 172.21.15.45:0/39697414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800f450 con 0x7fb1f010c110 2024-09-10T10:03:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f019d5c0 con 0x7fb1f010c110 2024-09-10T10:03:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1e77fe640 1 -- 172.21.15.45:0/39697414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e800fad0 con 0x7fb1f010c110 2024-09-10T10:03:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.086+0000 7fb1e77fe640 1 -- 172.21.15.45:0/39697414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8018c10 con 0x7fb1f010c110 2024-09-10T10:03:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.087+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a4e50 con 0x7fb1f010c110 2024-09-10T10:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1e77fe640 1 -- 172.21.15.45:0/39697414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb1e800f5f0 con 0x7fb1f010c110 2024-09-10T10:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 Processor -- start 2024-09-10T10:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 -- start start 2024-09-10T10:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e30aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e315770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e3180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56380e26f2f0 con 0x56380e315cb0 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56380e217270 con 0x56380e30aaf0 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56380e26b2b0 con 0x56380e30b410 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e3180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e3180a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47446/0 (socket says 172.21.15.45:47446) 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.090+0000 7fb1edd9b640 1 -- 172.21.15.45:0/3548877142 learned_addr learned my addr 172.21.15.45:0/3548877142 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e315770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e30aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1e77fe640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb1dc0a2e40 0x7fb1dc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1eed9d640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb1dc0a2e40 0x7fb1dc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1edd9b640 1 -- 172.21.15.45:0/3548877142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 msgr2=0x56380e315770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e315770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1edd9b640 1 -- 172.21.15.45:0/3548877142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 msgr2=0x56380e30aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e30aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1edd9b640 1 -- 172.21.15.45:0/3548877142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380e216340 con 0x56380e315cb0 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1e77fe640 1 -- 172.21.15.45:0/39697414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb1e80cfd30 con 0x7fb1f010c110 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e3180a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8013940 tx=0x7fb1d800b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1aa46fcbdcae22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1e4ff9640 1 -- 172.21.15.45:0/3548877142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8004020 con 0x56380e315cb0 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1e4ff9640 1 -- 172.21.15.45:0/3548877142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d800ee40 con 0x56380e315cb0 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1eed9d640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb1dc0a2e40 0x7fb1dc0a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80cd430 tx=0x7fb1e8091580 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.091+0000 7fb1e4ff9640 1 -- 172.21.15.45:0/3548877142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8003910 con 0x56380e315cb0 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 -- 172.21.15.45:0/3548877142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 msgr2=0x56380e3180a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e3180a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8013940 tx=0x7fb1d800b520 comp rx=0 tx=0).stop 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 -- 172.21.15.45:0/3548877142 shutdown_connections 2024-09-10T10:03:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e3180a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e315770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/3548877142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e30aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 -- 172.21.15.45:0/3548877142 >> 172.21.15.45:0/3548877142 conn(0x56380e307060 msgr2=0x56380e307440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 -- 172.21.15.45:0/3548877142 shutdown_connections 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.092+0000 7fb1f7e24740 1 -- 172.21.15.45:0/3548877142 wait complete. 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 -- start start 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 Processor -- start 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e312ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e3130e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 0x56380e313620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56380e318880 con 0x56380e30aaf0 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56380e1f8290 con 0x56380e315cb0 2024-09-10T10:03:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56380e318a00 con 0x56380e30b410 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1f7e24740 10 client.0 _ll_register_callbacks cb 0x56380e229140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e312ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e312ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47462/0 (socket says 172.21.15.45:47462) 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1edd9b640 1 -- 172.21.15.45:0/597900504 learned_addr learned my addr 172.21.15.45:0/597900504 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e3130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 0x56380e313620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.093+0000 7fb1edd9b640 1 -- 172.21.15.45:0/597900504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 msgr2=0x56380e3130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e3130e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1edd9b640 1 -- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 msgr2=0x56380e313620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 0x56380e313620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1edd9b640 1 -- 172.21.15.45:0/597900504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380df82210 con 0x56380e30aaf0 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e3130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e312ba0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800e720 tx=0x7fb1d80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1908eca1cdaba829 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/597900504 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800eb60 con 0x56380e30aaf0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/597900504 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb1d8005020 con 0x56380e30aaf0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d800ee40 con 0x56380e30aaf0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/597900504 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb1d8005220 con 0x56380e30aaf0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1f7e24740 10 client.15162 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56380e04e620 con 0x56380e30aaf0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/597900504 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d800ee40 con 0x56380e30aaf0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.094+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/597900504 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb1d8004ca0 con 0x56380e30aaf0 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1f7e24740 10 client.15162 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1f7e24740 10 client.15162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1f7e24740 20 client.15162 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e325ba0 0x56380e327f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1f7e24740 4 client.15162 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56380e27ca20 con 0x56380e325ba0 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e325ba0 0x56380e327f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e325ba0 0x56380e327f90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b71cefc4bd11e6f7 server_cookie=eaf1868881ac8016 in_seq=0 out_seq=0 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.095+0000 7fb1d5ffb640 10 client.15162 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/597900504 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56380e27ca20 con 0x56380e325ba0 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1d5ffb640 10 client.15162 handle_command_reply: tid=0 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 client.15162 shutdown 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 msgr2=0x56380e312ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e312ba0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800e720 tx=0x7fb1d80035f0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 shutdown_connections 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e325ba0 0x56380e327f90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 0x56380e313620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e3130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1d5ffb640 0 client.15162 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/597900504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e312ba0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 >> 172.21.15.45:0/597900504 conn(0x56380e307060 msgr2=0x56380e30cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 shutdown_connections 2024-09-10T10:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 1 -- 172.21.15.45:0/597900504 wait complete. 2024-09-10T10:03:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.096+0000 7fb1f7e24740 20 client.15162 trim_cache size 0 max 16384 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 Processor -- start 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 -- start start 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e311450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e313450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 0x56380e31e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ab750 con 0x56380e30aaf0 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01adff0 con 0x56380e315cb0 2024-09-10T10:03:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01abde0 con 0x56380e30b410 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e311450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e311450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47476/0 (socket says 172.21.15.45:47476) 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.111+0000 7fb1edd9b640 1 -- 172.21.15.45:0/1498705560 learned_addr learned my addr 172.21.15.45:0/1498705560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e313450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1edd9b640 1 -- 172.21.15.45:0/1498705560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 msgr2=0x56380e313450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e313450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1edd9b640 1 -- 172.21.15.45:0/1498705560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 msgr2=0x56380e31e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 0x56380e31e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1edd9b640 1 -- 172.21.15.45:0/1498705560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380e306e90 con 0x56380e30aaf0 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e311450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800ba10 tx=0x7fb1d8005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc58ae705316f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1d7fff640 1 -- 172.21.15.45:0/1498705560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d803d070 con 0x56380e30aaf0 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1d7fff640 1 -- 172.21.15.45:0/1498705560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d8004420 con 0x56380e30aaf0 2024-09-10T10:03:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.112+0000 7fb1d7fff640 1 -- 172.21.15.45:0/1498705560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800e960 con 0x56380e30aaf0 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- 172.21.15.45:0/1498705560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 msgr2=0x56380e311450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e311450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800ba10 tx=0x7fb1d8005fc0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- 172.21.15.45:0/1498705560 shutdown_connections 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e315cb0 0x56380e31e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e313450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/1498705560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e30aaf0 0x56380e311450 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- 172.21.15.45:0/1498705560 >> 172.21.15.45:0/1498705560 conn(0x56380e307060 msgr2=0x56380e314c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- 172.21.15.45:0/1498705560 shutdown_connections 2024-09-10T10:03:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- 172.21.15.45:0/1498705560 wait complete. 2024-09-10T10:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- start start 2024-09-10T10:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 Processor -- start 2024-09-10T10:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e32cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e32d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e38b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ac4b0 con 0x56380e315cb0 2024-09-10T10:03:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01ad920 con 0x56380e30aaf0 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01acb80 con 0x56380e30b410 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.113+0000 7fb1f7e24740 10 client.0 _ll_register_callbacks cb 0x7fb1d800be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e38b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1ee59c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e32d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e38b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47488/0 (socket says 172.21.15.45:47488) 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 -- 172.21.15.45:0/2159629324 learned_addr learned my addr 172.21.15.45:0/2159629324 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e32cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 -- 172.21.15.45:0/2159629324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 msgr2=0x56380e32d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e32d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 -- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 msgr2=0x56380e32cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e32cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 -- 172.21.15.45:0/2159629324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380e38bd80 con 0x56380e315cb0 2024-09-10T10:03:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1edd9b640 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e38b6e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8011630 tx=0x7fb1d8013600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbb7cde751a5f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/2159629324 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d803d040 con 0x56380e315cb0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/2159629324 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb1d80118a0 con 0x56380e315cb0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d8004420 con 0x56380e315cb0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/2159629324 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb1d800e960 con 0x56380e315cb0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1f7e24740 10 client.15168 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.114+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56380e3942f0 con 0x56380e315cb0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/2159629324 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d8004420 con 0x56380e315cb0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/2159629324 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb1d8035970 con 0x56380e315cb0 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1f7e24740 10 client.15168 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1f7e24740 10 client.15168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1f7e24740 20 client.15168 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e394950 0x56380e396d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1f7e24740 4 client.15168 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:03:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56380e276d70 con 0x56380e394950 2024-09-10T10:03:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.115+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e394950 0x56380e396d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.116+0000 7fb1d5ffb640 10 client.15168 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.116+0000 7fb1ee59c640 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e394950 0x56380e396d40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dadd60078c2509a8 server_cookie=8e2abdd6778200e0 in_seq=0 out_seq=0 2024-09-10T10:03:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.126+0000 7fb1d5ffb640 1 -- 172.21.15.45:0/2159629324 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56380e276d70 con 0x56380e394950 2024-09-10T10:03:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.126+0000 7fb1d5ffb640 10 client.15168 handle_command_reply: tid=0 2024-09-10T10:03:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.126+0000 7fb1f7e24740 1 client.15168 shutdown 2024-09-10T10:03:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 msgr2=0x56380e38b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e38b6e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8011630 tx=0x7fb1d8013600 comp rx=0 tx=0).stop 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 shutdown_connections 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56380e394950 0x56380e396d40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56380e315cb0 0x56380e38b6e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1d5ffb640 0 client.15168 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56380e30b410 0x56380e32d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 --2- 172.21.15.45:0/2159629324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56380e30aaf0 0x56380e32cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 >> 172.21.15.45:0/2159629324 conn(0x56380e307060 msgr2=0x56380e30c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 shutdown_connections 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 1 -- 172.21.15.45:0/2159629324 wait complete. 2024-09-10T10:03:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f7e24740 20 client.15168 trim_cache size 0 max 16384 2024-09-10T10:03:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:03:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:03:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:03:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:03:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb1dc0a2e40 msgr2=0x7fb1dc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb1dc0a2e40 0x7fb1dc0a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80cd430 tx=0x7fb1e8091580 comp rx=0 tx=0).stop 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f010c110 msgr2=0x7fb1f019cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f010c110 0x7fb1f019cba0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800cce0 tx=0x7fb1e800a620 comp rx=0 tx=0).stop 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 shutdown_connections 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1f010c110 0x7fb1f019cba0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.127+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb1f01035e0 0x7fb1f0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.128+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb1dc0a2e40 0x7fb1dc0a5300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.128+0000 7fb1f4faf640 1 --2- 172.21.15.45:0/39697414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1f0102cc0 0x7fb1f01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.128+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 >> 172.21.15.45:0/39697414 conn(0x7fb1f00fc6b0 msgr2=0x7fb1f00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.128+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 shutdown_connections 2024-09-10T10:03:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:28.128+0000 7fb1f4faf640 1 -- 172.21.15.45:0/39697414 wait complete. 2024-09-10T10:03:28.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:28 smithi045 ceph-mon[31114]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-10T10:03:29.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:28 smithi162 ceph-mon[34963]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-10T10:03:29.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:28 smithi047 ceph-mon[35244]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-09-10T10:03:31.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:30 smithi162 ceph-mon[34963]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:31.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:30 smithi045 ceph-mon[31114]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:31.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:30 smithi047 ceph-mon[35244]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:33.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:32 smithi162 ceph-mon[34963]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:33.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:32 smithi045 ceph-mon[31114]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:33.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:32 smithi047 ceph-mon[35244]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:03:35.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:34 smithi162 ceph-mon[34963]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:35.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:34 smithi045 ceph-mon[31114]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:35.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:34 smithi047 ceph-mon[35244]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:37.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:36 smithi162 ceph-mon[34963]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:37.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:36 smithi045 ceph-mon[31114]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:37.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:36 smithi047 ceph-mon[35244]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:39.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:38 smithi162 ceph-mon[34963]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:39.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:38 smithi045 ceph-mon[31114]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:39.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:38 smithi047 ceph-mon[35244]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:41.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:40 smithi162 ceph-mon[34963]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:41.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:40 smithi045 ceph-mon[31114]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:41.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:40 smithi047 ceph-mon[35244]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 -- 172.21.15.45:0/3841613216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d81069c0 msgr2=0x7fb5d8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 --2- 172.21.15.45:0/3841613216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d81069c0 0x7fb5d8106da0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009930 tx=0x7fb5c802a420 comp rx=0 tx=0).stop 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 -- 172.21.15.45:0/3841613216 shutdown_connections 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 --2- 172.21.15.45:0/3841613216 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d8101350 0x7fb5d810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 --2- 172.21.15.45:0/3841613216 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d81009b0 0x7fb5d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 --2- 172.21.15.45:0/3841613216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d81069c0 0x7fb5d8106da0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 -- 172.21.15.45:0/3841613216 >> 172.21.15.45:0/3841613216 conn(0x7fb5d8077990 msgr2=0x7fb5d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 -- 172.21.15.45:0/3841613216 shutdown_connections 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 -- 172.21.15.45:0/3841613216 wait complete. 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 Processor -- start 2024-09-10T10:03:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.219+0000 7fb5de966640 1 -- start start 2024-09-10T10:03:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5de966640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d81009b0 0x7fb5d8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5de966640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8101350 0x7fb5d8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5de966640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d81069c0 0x7fb5d819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5de966640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8110e00 con 0x7fb5d8101350 2024-09-10T10:03:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5de966640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8110c80 con 0x7fb5d81069c0 2024-09-10T10:03:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5de966640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8110f80 con 0x7fb5d81009b0 2024-09-10T10:03:41.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8101350 0x7fb5d8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8101350 0x7fb5d8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45072/0 (socket says 172.21.15.45:45072) 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 -- 172.21.15.45:0/1625889751 learned_addr learned my addr 172.21.15.45:0/1625889751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d81009b0 0x7fb5d8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d81069c0 0x7fb5d819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 -- 172.21.15.45:0/1625889751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d81009b0 msgr2=0x7fb5d8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d81009b0 0x7fb5d8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 -- 172.21.15.45:0/1625889751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d81069c0 msgr2=0x7fb5d819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d81069c0 0x7fb5d819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d77fe640 1 -- 172.21.15.45:0/1625889751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d819d320 con 0x7fb5d8101350 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.220+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d81009b0 0x7fb5d8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:41.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.221+0000 7fb5d77fe640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8101350 0x7fb5d8198870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c890 tx=0x7fb5cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.221+0000 7fb5d57fa640 1 -- 172.21.15.45:0/1625889751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc004220 con 0x7fb5d8101350 2024-09-10T10:03:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.221+0000 7fb5d57fa640 1 -- 172.21.15.45:0/1625889751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5cc0043c0 con 0x7fb5d8101350 2024-09-10T10:03:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.221+0000 7fb5d57fa640 1 -- 172.21.15.45:0/1625889751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0049e0 con 0x7fb5d8101350 2024-09-10T10:03:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.221+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d819d610 con 0x7fb5d8101350 2024-09-10T10:03:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.221+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d81a4ea0 con 0x7fb5d8101350 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.223+0000 7fb5d57fa640 1 -- 172.21.15.45:0/1625889751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fb5cc0079b0 con 0x7fb5d8101350 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.223+0000 7fb5e17dc740 1 Processor -- start 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5e17dc740 1 -- start start 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81cf9d0 0x55dfa81cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d57fa640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c00a2e40 0x7fb5c00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81df0c0 0x55dfa81e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dfa81342f0 con 0x55dfa81d02f0 2024-09-10T10:03:41.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dfa80dc270 con 0x55dfa81df0c0 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55dfa81302b0 con 0x55dfa81cf9d0 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d77fe640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c00a2e40 0x7fb5c00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45074/0 (socket says 172.21.15.45:45074) 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 -- 172.21.15.45:0/2739560722 learned_addr learned my addr 172.21.15.45:0/2739560722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d57fa640 1 -- 172.21.15.45:0/1625889751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb5cc0cf6f0 con 0x7fb5d8101350 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81cf9d0 0x55dfa81cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81df0c0 0x55dfa81e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 -- 172.21.15.45:0/2739560722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81cf9d0 msgr2=0x55dfa81cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81cf9d0 0x55dfa81cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 -- 172.21.15.45:0/2739560722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81df0c0 msgr2=0x55dfa81e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81df0c0 0x55dfa81e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5d7fff640 1 -- 172.21.15.45:0/2739560722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfa80db340 con 0x55dfa81d02f0 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.224+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81df0c0 0x55dfa81e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.225+0000 7fb5d77fe640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c00a2e40 0x7fb5c00a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0ccdf0 tx=0x7fb5cc013b80 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.225+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81deb80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8033000 tx=0x7fb5c8032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f5db55204f924c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.225+0000 7fb5be7fc640 1 -- 172.21.15.45:0/2739560722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8035030 con 0x55dfa81d02f0 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.225+0000 7fb5be7fc640 1 -- 172.21.15.45:0/2739560722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c80317b0 con 0x55dfa81d02f0 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.225+0000 7fb5be7fc640 1 -- 172.21.15.45:0/2739560722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80046c0 con 0x55dfa81d02f0 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 -- 172.21.15.45:0/2739560722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 msgr2=0x55dfa81deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81deb80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8033000 tx=0x7fb5c8032d60 comp rx=0 tx=0).stop 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 -- 172.21.15.45:0/2739560722 shutdown_connections 2024-09-10T10:03:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81df0c0 0x55dfa81e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81deb80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/2739560722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81cf9d0 0x55dfa81cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 -- 172.21.15.45:0/2739560722 >> 172.21.15.45:0/2739560722 conn(0x55dfa81cc060 msgr2=0x55dfa81cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 -- 172.21.15.45:0/2739560722 shutdown_connections 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 -- 172.21.15.45:0/2739560722 wait complete. 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 -- start start 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.226+0000 7fb5e17dc740 1 Processor -- start 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa81dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dfa81e1c90 con 0x55dfa81df0c0 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dfa80bd290 con 0x55dfa81cf9d0 2024-09-10T10:03:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55dfa81e1e10 con 0x55dfa81d02f0 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5e17dc740 10 client.0 _ll_register_callbacks cb 0x55dfa80ee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa81dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa81dcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45082/0 (socket says 172.21.15.45:45082) 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 -- 172.21.15.45:0/407207737 learned_addr learned my addr 172.21.15.45:0/407207737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 -- 172.21.15.45:0/407207737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 msgr2=0x55dfa81dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 -- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 msgr2=0x55dfa81dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81dc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 -- 172.21.15.45:0/407207737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfa7e47210 con 0x55dfa81df0c0 2024-09-10T10:03:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa81dcac0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0014eb0 tx=0x7fb5d0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dced5741505c869 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.227+0000 7fb5b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5b77fe640 1 -- 172.21.15.45:0/407207737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0011e90 con 0x55dfa81df0c0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c80317b0 con 0x55dfa81df0c0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5b77fe640 1 -- 172.21.15.45:0/407207737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb5d0014110 con 0x55dfa81df0c0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5b77fe640 1 -- 172.21.15.45:0/407207737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb5d0014310 con 0x55dfa81df0c0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 10 client.15180 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dfa7f13620 con 0x55dfa81df0c0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5b77fe640 1 -- 172.21.15.45:0/407207737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c80317b0 con 0x55dfa81df0c0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5b77fe640 1 -- 172.21.15.45:0/407207737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb5d0029470 con 0x55dfa81df0c0 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 10 client.15180 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 10 client.15180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 20 client.15180 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa81ef100 0x55dfa81f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 4 client.15180 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.228+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfa8141a20 con 0x55dfa81ef100 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.229+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa81ef100 0x55dfa81f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.229+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa81ef100 0x55dfa81f14f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05be646016a2aeb server_cookie=dc0b753d7851ce7a in_seq=0 out_seq=0 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.229+0000 7fb5b77fe640 10 client.15180 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5b77fe640 1 -- 172.21.15.45:0/407207737 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfa8141a20 con 0x55dfa81ef100 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5b77fe640 10 client.15180 handle_command_reply: tid=0 2024-09-10T10:03:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 client.15180 shutdown 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 msgr2=0x55dfa81dcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa81dcac0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0014eb0 tx=0x7fb5d0016040 comp rx=0 tx=0).stop 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 shutdown_connections 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa81ef100 0x55dfa81f14f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa81dcac0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5b77fe640 0 client.15180 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/407207737 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 >> 172.21.15.45:0/407207737 conn(0x55dfa81cc060 msgr2=0x55dfa81d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 shutdown_connections 2024-09-10T10:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 1 -- 172.21.15.45:0/407207737 wait complete. 2024-09-10T10:03:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.230+0000 7fb5e17dc740 20 client.15180 trim_cache size 0 max 16384 2024-09-10T10:03:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.244+0000 7fb5e17dc740 1 Processor -- start 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.244+0000 7fb5e17dc740 1 -- start start 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81df0c0 0x55dfa81f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81a9590 con 0x55dfa81d02f0 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81abe10 con 0x55dfa81cf9d0 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81a9c00 con 0x55dfa81df0c0 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81f6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45090/0 (socket says 172.21.15.45:45090) 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 -- 172.21.15.45:0/3512445401 learned_addr learned my addr 172.21.15.45:0/3512445401 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81df0c0 0x55dfa81f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 -- 172.21.15.45:0/3512445401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81df0c0 msgr2=0x55dfa81f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81df0c0 0x55dfa81f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 -- 172.21.15.45:0/3512445401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 msgr2=0x55dfa81ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 -- 172.21.15.45:0/3512445401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfa81f1b50 con 0x55dfa81d02f0 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81f6d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8033000 tx=0x7fb5c80053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fc7c4ee29a1f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.245+0000 7fb5bdffb640 1 -- 172.21.15.45:0/3512445401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8035040 con 0x55dfa81d02f0 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5bdffb640 1 -- 172.21.15.45:0/3512445401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c8004250 con 0x55dfa81d02f0 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5bdffb640 1 -- 172.21.15.45:0/3512445401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8003970 con 0x55dfa81d02f0 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3512445401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 msgr2=0x55dfa81f6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81f6d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8033000 tx=0x7fb5c80053c0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3512445401 shutdown_connections 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81df0c0 0x55dfa81f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81d02f0 0x55dfa81f6d80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3512445401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3512445401 >> 172.21.15.45:0/3512445401 conn(0x55dfa81cc060 msgr2=0x55dfa81dd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3512445401 shutdown_connections 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.246+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3512445401 wait complete. 2024-09-10T10:03:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 -- start start 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 Processor -- start 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81e3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa8258d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81aa2d0 con 0x55dfa81df0c0 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81ab740 con 0x55dfa81cf9d0 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81aa9a0 con 0x55dfa81d02f0 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5e17dc740 10 client.0 _ll_register_callbacks cb 0x7fb5d0009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5dcf03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa8258d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5d7fff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5dcf03640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa8258d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45102/0 (socket says 172.21.15.45:45102) 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5dcf03640 1 -- 172.21.15.45:0/3395145563 learned_addr learned my addr 172.21.15.45:0/3395145563 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.247+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5d7fff640 1 -- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 msgr2=0x55dfa81e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81e3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5d7fff640 1 -- 172.21.15.45:0/3395145563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 msgr2=0x55dfa8258d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa8258d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5d7fff640 1 -- 172.21.15.45:0/3395145563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfa81dbe10 con 0x55dfa81d02f0 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa8258d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5d7fff640 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81e3680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8006010 tx=0x7fb5c802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=943e0cf41a92fe22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5b77fe640 1 -- 172.21.15.45:0/3395145563 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8035040 con 0x55dfa81d02f0 2024-09-10T10:03:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5b77fe640 1 -- 172.21.15.45:0/3395145563 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb5c80316b0 con 0x55dfa81d02f0 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c8004250 con 0x55dfa81d02f0 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5b77fe640 1 -- 172.21.15.45:0/3395145563 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb5c8061050 con 0x55dfa81d02f0 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5e17dc740 10 client.24856 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.248+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55dfa825a120 con 0x55dfa81d02f0 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5b77fe640 1 -- 172.21.15.45:0/3395145563 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c8004250 con 0x55dfa81d02f0 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5b77fe640 1 -- 172.21.15.45:0/3395145563 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb5c8059020 con 0x55dfa81d02f0 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5e17dc740 10 client.24856 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5e17dc740 10 client.24856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5e17dc740 20 client.24856 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa825a3b0 0x55dfa825c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5e17dc740 4 client.24856 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:03:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dfa813bd70 con 0x55dfa825a3b0 2024-09-10T10:03:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa825a3b0 0x55dfa825c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5dcf03640 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa825a3b0 0x55dfa825c7a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f65b6233f9b4308 server_cookie=214cecd67d6edef6 in_seq=0 out_seq=0 2024-09-10T10:03:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.249+0000 7fb5b77fe640 10 client.24856 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5b77fe640 1 -- 172.21.15.45:0/3395145563 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dfa813bd70 con 0x55dfa825a3b0 2024-09-10T10:03:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5b77fe640 10 client.24856 handle_command_reply: tid=0 2024-09-10T10:03:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 client.24856 shutdown 2024-09-10T10:03:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 msgr2=0x55dfa81e3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81e3680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8006010 tx=0x7fb5c802a420 comp rx=0 tx=0).stop 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 shutdown_connections 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dfa825a3b0 0x55dfa825c7a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dfa81df0c0 0x55dfa8258d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dfa81d02f0 0x55dfa81e3680 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5b77fe640 0 client.24856 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 --2- 172.21.15.45:0/3395145563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dfa81cf9d0 0x55dfa81e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 >> 172.21.15.45:0/3395145563 conn(0x55dfa81cc060 msgr2=0x55dfa81dc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 shutdown_connections 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 1 -- 172.21.15.45:0/3395145563 wait complete. 2024-09-10T10:03:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.253+0000 7fb5e17dc740 20 client.24856 trim_cache size 0 max 16384 2024-09-10T10:03:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:03:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:03:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:03:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:03:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c00a2e40 msgr2=0x7fb5c00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c00a2e40 0x7fb5c00a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0ccdf0 tx=0x7fb5cc013b80 comp rx=0 tx=0).stop 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8101350 msgr2=0x7fb5d8198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8101350 0x7fb5d8198870 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c890 tx=0x7fb5cc00cd50 comp rx=0 tx=0).stop 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 shutdown_connections 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c00a2e40 0x7fb5c00a5300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d81069c0 0x7fb5d819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8101350 0x7fb5d8198870 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 --2- 172.21.15.45:0/1625889751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d81009b0 0x7fb5d8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 >> 172.21.15.45:0/1625889751 conn(0x7fb5d8077990 msgr2=0x7fb5d8104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 shutdown_connections 2024-09-10T10:03:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:41.254+0000 7fb5de966640 1 -- 172.21.15.45:0/1625889751 wait complete. 2024-09-10T10:03:43.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:42 smithi162 ceph-mon[34963]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:43.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:42 smithi045 ceph-mon[31114]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:43.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:42 smithi047 ceph-mon[35244]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 6 op/s 2024-09-10T10:03:45.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:44 smithi162 ceph-mon[34963]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-10T10:03:45.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:44 smithi045 ceph-mon[31114]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-10T10:03:45.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:44 smithi047 ceph-mon[35244]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-10T10:03:47.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:46 smithi162 ceph-mon[34963]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:47.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:46 smithi045 ceph-mon[31114]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:47.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:46 smithi047 ceph-mon[35244]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:49.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:48 smithi162 ceph-mon[34963]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:49.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:48 smithi045 ceph-mon[31114]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:49.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:48 smithi047 ceph-mon[35244]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:51.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:50 smithi162 ceph-mon[34963]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:51.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:50 smithi045 ceph-mon[31114]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:51.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:50 smithi047 ceph-mon[35244]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:53.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:52 smithi162 ceph-mon[34963]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:52 smithi045 ceph-mon[31114]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:53.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:52 smithi047 ceph-mon[35244]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 -- 172.21.15.45:0/653277884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94102470 msgr2=0x7f6e9410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 --2- 172.21.15.45:0/653277884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94102470 0x7f6e9410e9a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009cd0 tx=0x7f6e8c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 -- 172.21.15.45:0/653277884 shutdown_connections 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 --2- 172.21.15.45:0/653277884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94102470 0x7f6e9410e9a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 --2- 172.21.15.45:0/653277884 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e94101ad0 0x7f6e94101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 --2- 172.21.15.45:0/653277884 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e94107b70 0x7f6e94107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 -- 172.21.15.45:0/653277884 >> 172.21.15.45:0/653277884 conn(0x7f6e940fd7d0 msgr2=0x7f6e940ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 -- 172.21.15.45:0/653277884 shutdown_connections 2024-09-10T10:03:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.386+0000 7f6e9b998640 1 -- 172.21.15.45:0/653277884 wait complete. 2024-09-10T10:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.387+0000 7f6e9b998640 1 Processor -- start 2024-09-10T10:03:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.387+0000 7f6e9b998640 1 -- start start 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.387+0000 7f6e9b998640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e94101ad0 0x7f6e94198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.387+0000 7f6e9b998640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e94102470 0x7f6e94198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.387+0000 7f6e9b998640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94107b70 0x7f6e9419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e98f33640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e94102470 0x7f6e94198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99f35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94107b70 0x7f6e9419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99f35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94107b70 0x7f6e9419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50964/0 (socket says 172.21.15.45:50964) 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99f35640 1 -- 172.21.15.45:0/3149620474 learned_addr learned my addr 172.21.15.45:0/3149620474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94111f30 con 0x7f6e94107b70 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94111db0 con 0x7f6e94101ad0 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941120b0 con 0x7f6e94102470 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99734640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e94101ad0 0x7f6e94198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99734640 1 -- 172.21.15.45:0/3149620474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e94102470 msgr2=0x7f6e94198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99734640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e94102470 0x7f6e94198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99734640 1 -- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94107b70 msgr2=0x7f6e9419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99734640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94107b70 0x7f6e9419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99734640 1 -- 172.21.15.45:0/3149620474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9419d330 con 0x7f6e94101ad0 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e99734640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e94101ad0 0x7f6e94198340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009bc0 tx=0x7f6e9000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.388+0000 7f6e8a7fc640 1 -- 172.21.15.45:0/3149620474 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9001e070 con 0x7f6e94101ad0 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.389+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9419d620 con 0x7f6e94101ad0 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.389+0000 7f6e8a7fc640 1 -- 172.21.15.45:0/3149620474 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e900198f0 con 0x7f6e94101ad0 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.389+0000 7f6e8a7fc640 1 -- 172.21.15.45:0/3149620474 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e900187f0 con 0x7f6e94101ad0 2024-09-10T10:03:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.389+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e94074420 con 0x7f6e94101ad0 2024-09-10T10:03:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.392+0000 7f6e8a7fc640 1 -- 172.21.15.45:0/3149620474 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6e90021070 con 0x7f6e94101ad0 2024-09-10T10:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.392+0000 7f6e9e811740 1 Processor -- start 2024-09-10T10:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.392+0000 7f6e9e811740 1 -- start start 2024-09-10T10:03:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9145af0 0x5585e9145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9150770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e91530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e9e811740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5585e90aa2f0 con 0x5585e9145af0 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e9e811740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5585e9052270 con 0x5585e9146410 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e9e811740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5585e90a62b0 con 0x5585e9150cb0 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9145af0 0x5585e9145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9145af0 0x5585e9145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50972/0 (socket says 172.21.15.45:50972) 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 -- 172.21.15.45:0/309550115 learned_addr learned my addr 172.21.15.45:0/309550115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99734640 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e98f33640 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e91530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 -- 172.21.15.45:0/309550115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 msgr2=0x5585e91530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e91530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e8a7fc640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6e800a2e40 0x7f6e800a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 -- 172.21.15.45:0/309550115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 msgr2=0x5585e9150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e99f35640 1 -- 172.21.15.45:0/309550115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585e9051340 con 0x5585e9145af0 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.393+0000 7f6e98f33640 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e91530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.394+0000 7f6e8a7fc640 1 -- 172.21.15.45:0/3149620474 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f6e900d2c90 con 0x7f6e94101ad0 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.394+0000 7f6e99f35640 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9145af0 0x5585e9145ed0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009a50 tx=0x7f6e8c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8f47419ce42eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.394+0000 7f6e99f35640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6e800a2e40 0x7f6e800a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.395+0000 7f6e99f35640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6e800a2e40 0x7f6e800a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c050000 tx=0x7f6e8c04c000 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.395+0000 7f6e7f7fe640 1 -- 172.21.15.45:0/309550115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c053070 con 0x5585e9145af0 2024-09-10T10:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.395+0000 7f6e7f7fe640 1 -- 172.21.15.45:0/309550115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8c036040 con 0x5585e9145af0 2024-09-10T10:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.395+0000 7f6e7f7fe640 1 -- 172.21.15.45:0/309550115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c030c00 con 0x5585e9145af0 2024-09-10T10:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 -- 172.21.15.45:0/309550115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9145af0 msgr2=0x5585e9145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9145af0 0x5585e9145ed0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009a50 tx=0x7f6e8c005ef0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 -- 172.21.15.45:0/309550115 shutdown_connections 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e91530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 --2- 172.21.15.45:0/309550115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9145af0 0x5585e9145ed0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 -- 172.21.15.45:0/309550115 >> 172.21.15.45:0/309550115 conn(0x5585e9142060 msgr2=0x5585e9142440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 -- 172.21.15.45:0/309550115 shutdown_connections 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 -- 172.21.15.45:0/309550115 wait complete. 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 -- start start 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.396+0000 7f6e9e811740 1 Processor -- start 2024-09-10T10:03:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e914db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9146410 0x5585e914e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e914e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e9e811740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5585e91538d0 con 0x5585e9146410 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e9e811740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5585e9033290 con 0x5585e9145af0 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e9e811740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5585e9153a50 con 0x5585e9150cb0 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e9e811740 10 client.0 _ll_register_callbacks cb 0x5585e9064140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e98f33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9146410 0x5585e914e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e99734640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e914db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e98f33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9146410 0x5585e914e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50978/0 (socket says 172.21.15.45:50978) 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e98f33640 1 -- 172.21.15.45:0/2751993670 learned_addr learned my addr 172.21.15.45:0/2751993670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.397+0000 7f6e99f35640 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e914e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e99734640 1 -- 172.21.15.45:0/2751993670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 msgr2=0x5585e914e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e99734640 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e914e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e99734640 1 -- 172.21.15.45:0/2751993670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9146410 msgr2=0x5585e914e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e99734640 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9146410 0x5585e914e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e99734640 1 -- 172.21.15.45:0/2751993670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585e8dbd210 con 0x5585e9145af0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e98f33640 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9146410 0x5585e914e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e99734640 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e914db40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e900d0330 tx=0x7f6e9000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67822b7387de5c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e7cff9640 1 -- 172.21.15.45:0/2751993670 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9009ecf0 con 0x5585e9145af0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e8c036040 con 0x5585e9145af0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e7cff9640 1 -- 172.21.15.45:0/2751993670 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f6e900a31b0 con 0x5585e9145af0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e7cff9640 1 -- 172.21.15.45:0/2751993670 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e900b8050 con 0x5585e9145af0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.398+0000 7f6e9e811740 10 client.24938 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.399+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5585e8e89620 con 0x5585e9145af0 2024-09-10T10:03:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.399+0000 7f6e7cff9640 1 -- 172.21.15.45:0/2751993670 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8c036040 con 0x5585e9145af0 2024-09-10T10:03:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e7cff9640 1 -- 172.21.15.45:0/2751993670 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6e90019aa0 con 0x5585e9145af0 2024-09-10T10:03:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e9e811740 10 client.24938 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e9e811740 10 client.24938 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e9e811740 20 client.24938 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e9e811740 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e9161040 0x5585e9163500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e9e811740 4 client.24938 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585e90b7a20 con 0x5585e9161040 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e98f33640 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e9161040 0x5585e9163500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e98f33640 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e9161040 0x5585e9163500 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b0d06ce9c762ce server_cookie=cb5b9eebaad1d3a8 in_seq=0 out_seq=0 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.400+0000 7f6e7cff9640 10 client.24938 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e7cff9640 1 -- 172.21.15.45:0/2751993670 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585e90b7a20 con 0x5585e9161040 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e7cff9640 10 client.24938 handle_command_reply: tid=0 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 client.24938 shutdown 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 msgr2=0x5585e914db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e914db40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6e900d0330 tx=0x7f6e9000be60 comp rx=0 tx=0).stop 2024-09-10T10:03:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 shutdown_connections 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e9161040 0x5585e9163500 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9150cb0 0x5585e914e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9146410 0x5585e914e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 --2- 172.21.15.45:0/2751993670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e914db40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 >> 172.21.15.45:0/2751993670 conn(0x5585e9142060 msgr2=0x5585e9147d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.401+0000 7f6e7cff9640 0 client.24938 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.402+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 shutdown_connections 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.402+0000 7f6e9e811740 1 -- 172.21.15.45:0/2751993670 wait complete. 2024-09-10T10:03:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.402+0000 7f6e9e811740 20 client.24938 trim_cache size 0 max 16384 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 Processor -- start 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 -- start start 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9145af0 0x5585e914e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9168e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e9169350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941a9490 con 0x5585e9150cb0 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941abc40 con 0x5585e9146410 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.430+0000 7f6e9e811740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941a9a50 con 0x5585e9145af0 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e99f35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e9169350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e99f35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e9169350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50990/0 (socket says 172.21.15.45:50990) 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e99f35640 1 -- 172.21.15.45:0/831899528 learned_addr learned my addr 172.21.15.45:0/831899528 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e98f33640 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e99734640 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9145af0 0x5585e914e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e98f33640 1 -- 172.21.15.45:0/831899528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9145af0 msgr2=0x5585e914e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e98f33640 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9145af0 0x5585e914e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e98f33640 1 -- 172.21.15.45:0/831899528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 msgr2=0x5585e9169350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e98f33640 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e9169350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e98f33640 1 -- 172.21.15.45:0/831899528 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585e9141e90 con 0x5585e9146410 2024-09-10T10:03:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e99f35640 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e9169350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.431+0000 7f6e98f33640 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9168e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5585e9163d80 tx=0x7f6e84013410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d13e89b97f1302 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.432+0000 7f6e7effd640 1 -- 172.21.15.45:0/831899528 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84014c60 con 0x5585e9146410 2024-09-10T10:03:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.432+0000 7f6e7effd640 1 -- 172.21.15.45:0/831899528 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e84014e00 con 0x5585e9146410 2024-09-10T10:03:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.432+0000 7f6e7effd640 1 -- 172.21.15.45:0/831899528 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8401fa10 con 0x5585e9146410 2024-09-10T10:03:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 -- 172.21.15.45:0/831899528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 msgr2=0x5585e9168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9168e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x5585e9163d80 tx=0x7f6e84013410 comp rx=0 tx=0).stop 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 -- 172.21.15.45:0/831899528 shutdown_connections 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e9169350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9146410 0x5585e9168e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 --2- 172.21.15.45:0/831899528 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9145af0 0x5585e914e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 -- 172.21.15.45:0/831899528 >> 172.21.15.45:0/831899528 conn(0x5585e9142060 msgr2=0x5585e9148df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 -- 172.21.15.45:0/831899528 shutdown_connections 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 -- 172.21.15.45:0/831899528 wait complete. 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 -- start start 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.433+0000 7f6e9e811740 1 Processor -- start 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e9158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9146410 0x5585e9159220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e9e811740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e914a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e9e811740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941aa100 con 0x5585e9150cb0 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e9e811740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941ab570 con 0x5585e9145af0 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e9e811740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941aa7d0 con 0x5585e9146410 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e9158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e9158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:59128/0 (socket says 172.21.15.45:59128) 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 -- 172.21.15.45:0/761721335 learned_addr learned my addr 172.21.15.45:0/761721335 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e9e811740 10 client.0 _ll_register_callbacks cb 0x7f6e90010200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99f35640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e914a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e98f33640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9146410 0x5585e9159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 -- 172.21.15.45:0/761721335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9146410 msgr2=0x5585e9159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9146410 0x5585e9159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 -- 172.21.15.45:0/761721335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 msgr2=0x5585e914a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e914a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.441 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e99734640 1 -- 172.21.15.45:0/761721335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585e915f7b0 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.434+0000 7f6e98f33640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9146410 0x5585e9159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e99734640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e9158ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e9009d460 tx=0x7f6e900d08e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d180f1378ab887ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e7cff9640 1 -- 172.21.15.45:0/761721335 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9009eb10 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e84014e00 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e7cff9640 1 -- 172.21.15.45:0/761721335 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f6e900d0b40 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e7cff9640 1 -- 172.21.15.45:0/761721335 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6e9009ecf0 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e9e811740 10 client.24944 fetch_fsmap learned FSMap version 19 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5585e91cb2d0 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.435+0000 7f6e7cff9640 1 -- 172.21.15.45:0/761721335 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e900a34b0 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e7cff9640 1 -- 172.21.15.45:0/761721335 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6e9009e600 con 0x5585e9145af0 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e9e811740 10 client.24944 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e9e811740 10 client.24944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e9e811740 20 client.24944 populate_metadata read hostname 'smithi045' 2024-09-10T10:03:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e9e811740 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e91cb9e0 0x5585e91cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e9e811740 4 client.24944 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:03:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6e84014e00 con 0x5585e91cb9e0 2024-09-10T10:03:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.436+0000 7f6e98f33640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e91cb9e0 0x5585e91cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.437+0000 7f6e7cff9640 10 client.24944 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:54.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.437+0000 7f6e98f33640 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e91cb9e0 0x5585e91cddd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b9c4e6f777d6f5 server_cookie=d70756a46978a61b in_seq=0 out_seq=0 2024-09-10T10:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.445+0000 7f6e7cff9640 1 -- 172.21.15.45:0/761721335 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6e84014e00 con 0x5585e91cb9e0 2024-09-10T10:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.445+0000 7f6e7cff9640 10 client.24944 handle_command_reply: tid=0 2024-09-10T10:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 client.24944 shutdown 2024-09-10T10:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 msgr2=0x5585e9158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e9158ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e9009d460 tx=0x7f6e900d08e0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 shutdown_connections 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5585e91cb9e0 0x5585e91cddd0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e7cff9640 0 client.24944 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5585e9150cb0 0x5585e914a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5585e9146410 0x5585e9159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 --2- 172.21.15.45:0/761721335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5585e9145af0 0x5585e9158ce0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 >> 172.21.15.45:0/761721335 conn(0x5585e9142060 msgr2=0x5585e9147550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 shutdown_connections 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 1 -- 172.21.15.45:0/761721335 wait complete. 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.446+0000 7f6e9e811740 20 client.24944 trim_cache size 0 max 16384 2024-09-10T10:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6e800a2e40 msgr2=0x7f6e800a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6e800a2e40 0x7f6e800a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c050000 tx=0x7f6e8c04c000 comp rx=0 tx=0).stop 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e94101ad0 msgr2=0x7f6e94198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e94101ad0 0x7f6e94198340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009bc0 tx=0x7f6e9000b690 comp rx=0 tx=0).stop 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 shutdown_connections 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94107b70 0x7f6e9419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e94102470 0x7f6e94198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6e800a2e40 0x7f6e800a5300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 --2- 172.21.15.45:0/3149620474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e94101ad0 0x7f6e94198340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 >> 172.21.15.45:0/3149620474 conn(0x7f6e940fd7d0 msgr2=0x7f6e940fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 shutdown_connections 2024-09-10T10:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:03:54.447+0000 7f6e9b998640 1 -- 172.21.15.45:0/3149620474 wait complete. 2024-09-10T10:03:55.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:54 smithi162 ceph-mon[34963]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T10:03:55.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:54 smithi045 ceph-mon[31114]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T10:03:55.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:54 smithi047 ceph-mon[35244]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T10:03:57.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:56 smithi162 ceph-mon[34963]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-10T10:03:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:56 smithi045 ceph-mon[31114]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-10T10:03:57.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:56 smithi047 ceph-mon[35244]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 938 B/s wr, 1 op/s 2024-09-10T10:03:59.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:03:58 smithi162 ceph-mon[34963]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T10:03:59.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:03:58 smithi045 ceph-mon[31114]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T10:03:59.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:03:58 smithi047 ceph-mon[35244]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T10:04:00.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:04:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:04:00.056+0000 7fc61f055640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:00.400 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:04:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:04:00.056+0000 7fc61f055640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:00.401 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:04:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:04:00.056+0000 7fc61f055640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:00.401 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:04:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:04:00.056+0000 7fc61f055640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:00.401 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:04:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:04:00.056+0000 7fc61f055640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T10:04:01.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:01 smithi162 ceph-mon[34963]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T10:04:01.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:01 smithi045 ceph-mon[31114]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T10:04:01.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:01 smithi047 ceph-mon[35244]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T10:04:03.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:03 smithi162 ceph-mon[34963]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T10:04:03.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:03 smithi162 ceph-mon[34963]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-09-10T10:04:03.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:03 smithi045 ceph-mon[31114]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T10:04:03.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:03 smithi045 ceph-mon[31114]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-09-10T10:04:03.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:03 smithi047 ceph-mon[35244]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T10:04:03.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:03 smithi047 ceph-mon[35244]: mgrmap e24: x(active, since 50s), standbys: z, y 2024-09-10T10:04:05.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:05 smithi162 ceph-mon[34963]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-09-10T10:04:05.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:05 smithi045 ceph-mon[31114]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-09-10T10:04:05.412 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:05 smithi047 ceph-mon[35244]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-09-10T10:04:07.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:07 smithi162 ceph-mon[34963]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-10T10:04:07.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:07 smithi045 ceph-mon[31114]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-10T10:04:07.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:07 smithi047 ceph-mon[35244]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-10T10:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 -- 172.21.15.45:0/2880462844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2810e430 msgr2=0x7fae28110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 --2- 172.21.15.45:0/2880462844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2810e430 0x7fae28110820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fae20009ca0 tx=0x7fae2002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 -- 172.21.15.45:0/2880462844 shutdown_connections 2024-09-10T10:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 --2- 172.21.15.45:0/2880462844 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae2810e430 0x7fae28110820 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 --2- 172.21.15.45:0/2880462844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae28101230 0x7fae2810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 --2- 172.21.15.45:0/2880462844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae28100880 0x7fae28100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 -- 172.21.15.45:0/2880462844 >> 172.21.15.45:0/2880462844 conn(0x7fae28077990 msgr2=0x7fae280fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 -- 172.21.15.45:0/2880462844 shutdown_connections 2024-09-10T10:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.537+0000 7fae30331640 1 -- 172.21.15.45:0/2880462844 wait complete. 2024-09-10T10:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 Processor -- start 2024-09-10T10:04:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 -- start start 2024-09-10T10:04:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae28100880 0x7fae2819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae28101230 0x7fae2819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae2810e430 0x7fae2819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae281140e0 con 0x7fae2810e430 2024-09-10T10:04:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae28113f60 con 0x7fae28100880 2024-09-10T10:04:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2e8ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae2810e430 0x7fae2819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2d8cc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae28101230 0x7fae2819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2e8ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae2810e430 0x7fae2819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44506/0 (socket says 172.21.15.45:44506) 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2e8ce640 1 -- 172.21.15.45:0/3817817478 learned_addr learned my addr 172.21.15.45:0/3817817478 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.538+0000 7fae30331640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fae28114260 con 0x7fae28101230 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae28100880 0x7fae2819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2d8cc640 1 -- 172.21.15.45:0/3817817478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae28100880 msgr2=0x7fae2819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae28100880 0x7fae2819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2d8cc640 1 -- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae2810e430 msgr2=0x7fae2819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae2810e430 0x7fae2819edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2d8cc640 1 -- 172.21.15.45:0/3817817478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2819f520 con 0x7fae28101230 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae28101230 0x7fae2819aa70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e9f0 tx=0x7fae1800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae1f7fe640 1 -- 172.21.15.45:0/3817817478 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1800ccc0 con 0x7fae28101230 2024-09-10T10:04:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae1f7fe640 1 -- 172.21.15.45:0/3817817478 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18010910 con 0x7fae28101230 2024-09-10T10:04:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.539+0000 7fae1f7fe640 1 -- 172.21.15.45:0/3817817478 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18018960 con 0x7fae28101230 2024-09-10T10:04:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.540+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2819f810 con 0x7fae28101230 2024-09-10T10:04:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.540+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae281a70a0 con 0x7fae28101230 2024-09-10T10:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.542+0000 7fae1f7fe640 1 -- 172.21.15.45:0/3817817478 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fae18010ac0 con 0x7fae28101230 2024-09-10T10:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.543+0000 7fae1f7fe640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fae140a2fc0 0x7fae140a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.543+0000 7fae330cf740 1 Processor -- start 2024-09-10T10:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.543+0000 7fae330cf740 1 -- start start 2024-09-10T10:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.543+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fae140a2fc0 0x7fae140a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.543+0000 7fae1f7fe640 1 -- 172.21.15.45:0/3817817478 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fae180cfa80 con 0x7fae28101230 2024-09-10T10:04:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae330cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f31af0 0x562540f31ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae330cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae330cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f3f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae330cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562540e962f0 con 0x562540f32410 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae330cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562540e3e270 con 0x562540f3ccb0 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae330cf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562540e922b0 con 0x562540f31af0 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44518/0 (socket says 172.21.15.45:44518) 2024-09-10T10:04:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 -- 172.21.15.45:0/1271835685 learned_addr learned my addr 172.21.15.45:0/1271835685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f3f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 -- 172.21.15.45:0/1271835685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f31af0 msgr2=0x562540f31ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f31af0 0x562540f31ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 -- 172.21.15.45:0/1271835685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 msgr2=0x562540f3f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f3f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2d8cc640 1 -- 172.21.15.45:0/1271835685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562540e3d340 con 0x562540f32410 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.544+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f3f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.545+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fae140a2fc0 0x7fae140a5480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fae240095f0 tx=0x7fae24005f90 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.545+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3c770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fae1809ba20 tx=0x7fae1809c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972352e8c3a598ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.545+0000 7fae1cbfe640 1 -- 172.21.15.45:0/1271835685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180a4990 con 0x562540f32410 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.545+0000 7fae1cbfe640 1 -- 172.21.15.45:0/1271835685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18098390 con 0x562540f32410 2024-09-10T10:04:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.545+0000 7fae1cbfe640 1 -- 172.21.15.45:0/1271835685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18018b00 con 0x562540f32410 2024-09-10T10:04:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 -- 172.21.15.45:0/1271835685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 msgr2=0x562540f3c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3c770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fae1809ba20 tx=0x7fae1809c4e0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 -- 172.21.15.45:0/1271835685 shutdown_connections 2024-09-10T10:04:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f3f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3c770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 --2- 172.21.15.45:0/1271835685 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f31af0 0x562540f31ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 -- 172.21.15.45:0/1271835685 >> 172.21.15.45:0/1271835685 conn(0x562540f2e060 msgr2=0x562540f2e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 -- 172.21.15.45:0/1271835685 shutdown_connections 2024-09-10T10:04:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 -- 172.21.15.45:0/1271835685 wait complete. 2024-09-10T10:04:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 -- start start 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.546+0000 7fae330cf740 1 Processor -- start 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae330cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f39bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae330cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae330cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f3ccb0 0x562540f3a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae330cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562540f3f880 con 0x562540f32410 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae330cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562540e1f290 con 0x562540f31af0 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae330cf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562540f3fa00 con 0x562540f3ccb0 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae330cf740 10 client.0 _ll_register_callbacks cb 0x562540e50140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44530/0 (socket says 172.21.15.45:44530) 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 -- 172.21.15.45:0/2693650014 learned_addr learned my addr 172.21.15.45:0/2693650014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f39bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f3ccb0 0x562540f3a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 -- 172.21.15.45:0/2693650014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f3ccb0 msgr2=0x562540f3a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f3ccb0 0x562540f3a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 -- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 msgr2=0x562540f39bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f39bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2d8cc640 1 -- 172.21.15.45:0/2693650014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562540ba9210 con 0x562540f32410 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.547+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f3ccb0 0x562540f3a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3a0f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e9c0 tx=0x7fae18009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1f43d9ccc3f087 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae11ffb640 1 -- 172.21.15.45:0/2693650014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1807f060 con 0x562540f32410 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae11ffb640 1 -- 172.21.15.45:0/2693650014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fae18010640 con 0x562540f32410 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae18098390 con 0x562540f32410 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae11ffb640 1 -- 172.21.15.45:0/2693650014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fae180ba070 con 0x562540f32410 2024-09-10T10:04:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae330cf740 10 client.15222 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562540c75620 con 0x562540f32410 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.548+0000 7fae11ffb640 1 -- 172.21.15.45:0/2693650014 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18098390 con 0x562540f32410 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae11ffb640 1 -- 172.21.15.45:0/2693650014 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fae18018b00 con 0x562540f32410 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae330cf740 10 client.15222 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae330cf740 10 client.15222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae330cf740 20 client.15222 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae330cf740 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540f4cab0 0x562540f4eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae330cf740 4 client.15222 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562540ea3a20 con 0x562540f4cab0 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.549+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540f4cab0 0x562540f4eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.550+0000 7fae11ffb640 10 client.15222 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.550+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540f4cab0 0x562540f4eea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4850c6c43f474a server_cookie=86c9865115fe9c37 in_seq=0 out_seq=0 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae11ffb640 1 -- 172.21.15.45:0/2693650014 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562540ea3a20 con 0x562540f4cab0 2024-09-10T10:04:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae11ffb640 10 client.15222 handle_command_reply: tid=0 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 client.15222 shutdown 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 msgr2=0x562540f3a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3a0f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e9c0 tx=0x7fae18009700 comp rx=0 tx=0).stop 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 shutdown_connections 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540f4cab0 0x562540f4eea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f3ccb0 0x562540f3a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f32410 0x562540f3a0f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 --2- 172.21.15.45:0/2693650014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f39bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae11ffb640 0 client.15222 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 >> 172.21.15.45:0/2693650014 conn(0x562540f2e060 msgr2=0x562540f33d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 shutdown_connections 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 1 -- 172.21.15.45:0/2693650014 wait complete. 2024-09-10T10:04:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.551+0000 7fae330cf740 20 client.15222 trim_cache size 0 max 16384 2024-09-10T10:04:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.565+0000 7fae330cf740 1 Processor -- start 2024-09-10T10:04:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.565+0000 7fae330cf740 1 -- start start 2024-09-10T10:04:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae330cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f3b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae330cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f3a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae330cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f3ccb0 0x562540f54440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae330cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae281adaa0 con 0x562540f3ccb0 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae330cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae281b0340 con 0x562540f31af0 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae330cf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ae130 con 0x562540f32410 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f3ccb0 0x562540f54440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f3ccb0 0x562540f54440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44536/0 (socket says 172.21.15.45:44536) 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 -- 172.21.15.45:0/2572503139 learned_addr learned my addr 172.21.15.45:0/2572503139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e0cd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f3b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f3a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 -- 172.21.15.45:0/2572503139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 msgr2=0x562540f3a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f3a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 -- 172.21.15.45:0/2572503139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 msgr2=0x562540f3b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f3b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e8ce640 1 -- 172.21.15.45:0/2572503139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562540f2de90 con 0x562540f3ccb0 2024-09-10T10:04:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.566+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f3a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.567+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f3ccb0 0x562540f54440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae2002ade0 tx=0x7fae2002f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28b06614077879c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.567+0000 7fae13fff640 1 -- 172.21.15.45:0/2572503139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2004a070 con 0x562540f3ccb0 2024-09-10T10:04:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.567+0000 7fae13fff640 1 -- 172.21.15.45:0/2572503139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae20036050 con 0x562540f3ccb0 2024-09-10T10:04:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.567+0000 7fae13fff640 1 -- 172.21.15.45:0/2572503139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20004040 con 0x562540f3ccb0 2024-09-10T10:04:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 -- 172.21.15.45:0/2572503139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f3ccb0 msgr2=0x562540f54440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f3ccb0 0x562540f54440 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae2002ade0 tx=0x7fae2002f450 comp rx=0 tx=0).stop 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 -- 172.21.15.45:0/2572503139 shutdown_connections 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f3ccb0 0x562540f54440 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f3a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 --2- 172.21.15.45:0/2572503139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f31af0 0x562540f3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 -- 172.21.15.45:0/2572503139 >> 172.21.15.45:0/2572503139 conn(0x562540f2e060 msgr2=0x562540f38b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 -- 172.21.15.45:0/2572503139 shutdown_connections 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 -- 172.21.15.45:0/2572503139 wait complete. 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 -- start start 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 Processor -- start 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.568+0000 7fae330cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f31af0 0x562540f453b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f31af0 0x562540f453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae330cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f36860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae330cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f36da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae330cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae281ae800 con 0x562540f31af0 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f31af0 0x562540f453b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44550/0 (socket says 172.21.15.45:44550) 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 -- 172.21.15.45:0/4037636059 learned_addr learned my addr 172.21.15.45:0/4037636059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae281afc70 con 0x562540f3ccb0 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fae281aeed0 con 0x562540f32410 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae330cf740 10 client.0 _ll_register_callbacks cb 0x7fae18005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e8ce640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f36da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f36860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 -- 172.21.15.45:0/4037636059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 msgr2=0x562540f36860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f36860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 -- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 msgr2=0x562540f36da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f36da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 -- 172.21.15.45:0/4037636059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562540f292d0 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f36860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae2e0cd640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f31af0 0x562540f453b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x562540f3fb10 tx=0x7fae24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2598c11e0937dee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae11ffb640 1 -- 172.21.15.45:0/4037636059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2402c070 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.569+0000 7fae11ffb640 1 -- 172.21.15.45:0/4037636059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fae24005220 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae20036050 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae11ffb640 1 -- 172.21.15.45:0/4037636059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fae24005420 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae330cf740 10 client.15228 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562540fb7260 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae11ffb640 1 -- 172.21.15.45:0/4037636059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae20036050 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae11ffb640 1 -- 172.21.15.45:0/4037636059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fae2401fa20 con 0x562540f31af0 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae330cf740 10 client.15228 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae330cf740 10 client.15228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae330cf740 20 client.15228 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.570+0000 7fae330cf740 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540fb7a60 0x562540fb9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.571+0000 7fae330cf740 4 client.15228 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:04:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.571+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562540e9dd70 con 0x562540fb7a60 2024-09-10T10:04:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.571+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540fb7a60 0x562540fb9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.571+0000 7fae11ffb640 10 client.15228 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.571+0000 7fae2d8cc640 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540fb7a60 0x562540fb9e80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94fedc65fc764bdc server_cookie=c353b27bca8a230b in_seq=0 out_seq=0 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae11ffb640 1 -- 172.21.15.45:0/4037636059 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562540e9dd70 con 0x562540fb7a60 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae11ffb640 10 client.15228 handle_command_reply: tid=0 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 client.15228 shutdown 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f31af0 msgr2=0x562540f453b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f31af0 0x562540f453b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x562540f3fb10 tx=0x7fae24004730 comp rx=0 tx=0).stop 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 shutdown_connections 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562540fb7a60 0x562540fb9e80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562540f3ccb0 0x562540f36da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562540f32410 0x562540f36860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae11ffb640 0 client.15228 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 --2- 172.21.15.45:0/4037636059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562540f31af0 0x562540f453b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 >> 172.21.15.45:0/4037636059 conn(0x562540f2e060 msgr2=0x562540f38330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 shutdown_connections 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 1 -- 172.21.15.45:0/4037636059 wait complete. 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.577+0000 7fae330cf740 20 client.15228 trim_cache size 0 max 16384 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:04:07.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:04:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fae140a2fc0 msgr2=0x7fae140a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fae140a2fc0 0x7fae140a5480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fae240095f0 tx=0x7fae24005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae28101230 msgr2=0x7fae2819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae28101230 0x7fae2819aa70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae1800e9f0 tx=0x7fae1800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 shutdown_connections 2024-09-10T10:04:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae2810e430 0x7fae2819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fae28101230 0x7fae2819aa70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fae140a2fc0 0x7fae140a5480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 --2- 172.21.15.45:0/3817817478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae28100880 0x7fae2819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 >> 172.21.15.45:0/3817817478 conn(0x7fae28077990 msgr2=0x7fae280ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 shutdown_connections 2024-09-10T10:04:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:07.578+0000 7fae30331640 1 -- 172.21.15.45:0/3817817478 wait complete. 2024-09-10T10:04:09.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:09 smithi162 ceph-mon[34963]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-09-10T10:04:09.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:09 smithi045 ceph-mon[31114]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-09-10T10:04:09.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:09 smithi047 ceph-mon[35244]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-09-10T10:04:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:11 smithi162 ceph-mon[34963]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-10T10:04:11.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:11 smithi045 ceph-mon[31114]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-10T10:04:11.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:11 smithi047 ceph-mon[35244]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-10T10:04:13.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:13 smithi162 ceph-mon[34963]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-10T10:04:13.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:13 smithi045 ceph-mon[31114]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-10T10:04:13.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:13 smithi047 ceph-mon[35244]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-09-10T10:04:14.461 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 92% (11697/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T10:04:15.159 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:15.160 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:04:15.164 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T10:04:15.232 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T10:04:15.235 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:04:15.235 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T10:04:15.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:15 smithi162 ceph-mon[34963]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-10T10:04:15.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:15 smithi045 ceph-mon[31114]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-10T10:04:15.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:15 smithi047 ceph-mon[35244]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-10T10:04:15.968 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T10:04:16.042 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T10:04:16.086 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T10:04:16.088 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:04:16.088 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T10:04:16.128 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T10:04:16.131 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T10:04:16.132 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T10:04:16.189 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-10T10:04:16.190 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-10T10:04:16.190 DEBUG:teuthology.orchestra.run.smithi045:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-10T10:04:16.251 INFO:tasks.workunit.client.0.smithi045.stdout:getting iogen 2024-09-10T10:04:16.251 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'getting iogen' 2024-09-10T10:04:16.252 INFO:tasks.workunit.client.0.smithi045.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-10T10:04:16.274 INFO:tasks.workunit.client.0.smithi045.stderr:--2024-09-10 10:04:16-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-10T10:04:16.275 INFO:tasks.workunit.client.0.smithi045.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-10T10:04:16.296 INFO:tasks.workunit.client.0.smithi045.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-10T10:04:16.301 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:16.317 INFO:tasks.workunit.client.0.smithi045.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-10T10:04:16.317 INFO:tasks.workunit.client.0.smithi045.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-10T10:04:16.318 INFO:tasks.workunit.client.0.smithi045.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-10T10:04:16.321 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:16.321 INFO:tasks.workunit.client.0.smithi045.stderr: 0K ....... 100% 1.20G=0s 2024-09-10T10:04:16.321 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:16.321 INFO:tasks.workunit.client.0.smithi045.stderr:2024-09-10 10:04:16 (1.20 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-10T10:04:16.321 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-09-10T10:04:16.321 INFO:tasks.workunit.client.0.smithi045.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-10T10:04:16.334 INFO:tasks.workunit.client.0.smithi045.stdout:iogen_3.1p0 2024-09-10T10:04:16.335 INFO:tasks.workunit.client.0.smithi045.stdout:iogen_3.1p0/src 2024-09-10T10:04:16.335 INFO:tasks.workunit.client.0.smithi045.stdout:iogen_3.1p0/src/iogen.8 2024-09-10T10:04:16.336 INFO:tasks.workunit.client.0.smithi045.stdout:iogen_3.1p0/src/iogen.c 2024-09-10T10:04:16.337 INFO:tasks.workunit.client.0.smithi045.stdout:iogen_3.1p0/README 2024-09-10T10:04:16.338 INFO:tasks.workunit.client.0.smithi045.stdout:iogen_3.1p0/Makefile.bsd 2024-09-10T10:04:16.338 INFO:tasks.workunit.client.0.smithi045.stdout:iogen_3.1p0/Makefile 2024-09-10T10:04:16.339 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd iogen_3.1p0 2024-09-10T10:04:16.340 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'making iogen' 2024-09-10T10:04:16.340 INFO:tasks.workunit.client.0.smithi045.stdout:making iogen 2024-09-10T10:04:16.340 INFO:tasks.workunit.client.0.smithi045.stderr:+ make 2024-09-10T10:04:16.349 INFO:tasks.workunit.client.0.smithi045.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-10T10:04:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:16.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:16.604 INFO:tasks.workunit.client.0.smithi045.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-10T10:04:16.713 INFO:tasks.workunit.client.0.smithi045.stdout:running iogen 2024-09-10T10:04:16.713 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'running iogen' 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:+ ./iogen -n 5 -s 2g 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:child spawned 85039 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:child spawned 85040 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:child spawned 85041 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:child spawned 85042 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:child spawned 85043 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'sleep for 10 min' 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stdout:sleep for 10 min 2024-09-10T10:04:16.714 INFO:tasks.workunit.client.0.smithi045.stderr:+ sleep 600 2024-09-10T10:04:17.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:17 smithi045 ceph-mon[31114]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-09-10T10:04:17.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:17.033 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:17.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:17 smithi162 ceph-mon[34963]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-09-10T10:04:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:04:17.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:04:17.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:17 smithi047 ceph-mon[35244]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 597 B/s wr, 2 op/s 2024-09-10T10:04:17.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:17.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:17.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:04:19.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:19 smithi162 ceph-mon[34963]: pgmap v36: 97 pgs: 97 active+clean; 319 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-09-10T10:04:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:19 smithi045 ceph-mon[31114]: pgmap v36: 97 pgs: 97 active+clean; 319 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-09-10T10:04:19.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:19 smithi047 ceph-mon[35244]: pgmap v36: 97 pgs: 97 active+clean; 319 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 26 MiB/s wr, 34 op/s 2024-09-10T10:04:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.738+0000 7fc69a477640 1 -- 172.21.15.45:0/1714358180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc694103b00 msgr2=0x7fc694103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.738+0000 7fc69a477640 1 --2- 172.21.15.45:0/1714358180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc694103b00 0x7fc694103f60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc684009930 tx=0x7fc68402a420 comp rx=0 tx=0).stop 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.739+0000 7fc69a477640 1 -- 172.21.15.45:0/1714358180 shutdown_connections 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.739+0000 7fc69a477640 1 --2- 172.21.15.45:0/1714358180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6941044a0 0x7fc6941109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.739+0000 7fc69a477640 1 --2- 172.21.15.45:0/1714358180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc694103b00 0x7fc694103f60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.739+0000 7fc69a477640 1 --2- 172.21.15.45:0/1714358180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc694109ba0 0x7fc694109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.739+0000 7fc69a477640 1 -- 172.21.15.45:0/1714358180 >> 172.21.15.45:0/1714358180 conn(0x7fc6940ff7c0 msgr2=0x7fc694101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.739+0000 7fc69a477640 1 -- 172.21.15.45:0/1714358180 shutdown_connections 2024-09-10T10:04:20.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.739+0000 7fc69a477640 1 -- 172.21.15.45:0/1714358180 wait complete. 2024-09-10T10:04:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.741+0000 7fc69a477640 1 Processor -- start 2024-09-10T10:04:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.741+0000 7fc69a477640 1 -- start start 2024-09-10T10:04:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc69a477640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc694103b00 0x7fc69419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc69a477640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6941044a0 0x7fc69419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc69a477640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc694109ba0 0x7fc69419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc69a477640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc694113fb0 con 0x7fc6941044a0 2024-09-10T10:04:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc69a477640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc694113e30 con 0x7fc694103b00 2024-09-10T10:04:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc69a477640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc694114130 con 0x7fc694109ba0 2024-09-10T10:04:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc698c74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6941044a0 0x7fc69419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc698c74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6941044a0 0x7fc69419aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50814/0 (socket says 172.21.15.45:50814) 2024-09-10T10:04:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.742+0000 7fc698c74640 1 -- 172.21.15.45:0/3532465059 learned_addr learned my addr 172.21.15.45:0/3532465059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.743+0000 7fc698c74640 1 -- 172.21.15.45:0/3532465059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc694109ba0 msgr2=0x7fc69419ede0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.743+0000 7fc698c74640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc694109ba0 0x7fc69419ede0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.743+0000 7fc698c74640 1 -- 172.21.15.45:0/3532465059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc694103b00 msgr2=0x7fc69419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.743+0000 7fc698c74640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc694103b00 0x7fc69419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.743+0000 7fc698c74640 1 -- 172.21.15.45:0/3532465059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69419f510 con 0x7fc6941044a0 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.743+0000 7fc698c74640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6941044a0 0x7fc69419aa60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc684006010 tx=0x7fc684032b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.744+0000 7fc68a7fc640 1 -- 172.21.15.45:0/3532465059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68403f030 con 0x7fc6941044a0 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.744+0000 7fc68a7fc640 1 -- 172.21.15.45:0/3532465059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc684003ad0 con 0x7fc6941044a0 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.744+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69419f7a0 con 0x7fc6941044a0 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.744+0000 7fc68a7fc640 1 -- 172.21.15.45:0/3532465059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684035070 con 0x7fc6941044a0 2024-09-10T10:04:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.744+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a7090 con 0x7fc6941044a0 2024-09-10T10:04:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.749+0000 7fc69e46f740 1 Processor -- start 2024-09-10T10:04:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.749+0000 7fc69e46f740 1 -- start start 2024-09-10T10:04:20.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.750+0000 7fc68a7fc640 1 -- 172.21.15.45:0/3532465059 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fc684042030 con 0x7fc6941044a0 2024-09-10T10:04:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.750+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa8a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.750+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa94940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.750+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.752 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc69e46f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d6f9ee2f0 con 0x563d6fa94e80 2024-09-10T10:04:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc69e46f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563d6f996270 con 0x563d6fa89cc0 2024-09-10T10:04:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc69e46f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563d6f9ea2b0 con 0x563d6fa8a5e0 2024-09-10T10:04:20.753 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc68a7fc640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc6800a2f10 0x7fc6800a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa97270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50820/0 (socket says 172.21.15.45:50820) 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 -- 172.21.15.45:0/3193230677 learned_addr learned my addr 172.21.15.45:0/3193230677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc699c76640 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa8a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 -- 172.21.15.45:0/3193230677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 msgr2=0x563d6fa94940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa94940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 -- 172.21.15.45:0/3193230677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 msgr2=0x563d6fa8a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa8a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.751+0000 7fc698c74640 1 -- 172.21.15.45:0/3193230677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6f995340 con 0x563d6fa94e80 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.752+0000 7fc698c74640 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa97270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc6840ba040 tx=0x7fc6840ba8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84770f66b74e3a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.752+0000 7fc68a7fc640 1 -- 172.21.15.45:0/3532465059 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fc6840f0060 con 0x7fc6941044a0 2024-09-10T10:04:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.752+0000 7fc699c76640 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa8a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.752+0000 7fc67f7fe640 1 -- 172.21.15.45:0/3193230677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684049030 con 0x563d6fa94e80 2024-09-10T10:04:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.752+0000 7fc67f7fe640 1 -- 172.21.15.45:0/3193230677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6840bcad0 con 0x563d6fa94e80 2024-09-10T10:04:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.752+0000 7fc67f7fe640 1 -- 172.21.15.45:0/3193230677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840bc3d0 con 0x563d6fa94e80 2024-09-10T10:04:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.752+0000 7fc699c76640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc6800a2f10 0x7fc6800a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.754+0000 7fc69e46f740 1 -- 172.21.15.45:0/3193230677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 msgr2=0x563d6fa97270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.754+0000 7fc69e46f740 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa97270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc6840ba040 tx=0x7fc6840ba8a0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.754+0000 7fc69e46f740 1 -- 172.21.15.45:0/3193230677 shutdown_connections 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.754+0000 7fc69e46f740 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa97270 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.754+0000 7fc69e46f740 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa94940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.754+0000 7fc69e46f740 1 --2- 172.21.15.45:0/3193230677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa8a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.754+0000 7fc69e46f740 1 -- 172.21.15.45:0/3193230677 >> 172.21.15.45:0/3193230677 conn(0x563d6fa86230 msgr2=0x563d6fa86610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.755+0000 7fc699c76640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc6800a2f10 0x7fc6800a53d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e10 tx=0x7fc68c004a40 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.755+0000 7fc69e46f740 1 -- 172.21.15.45:0/3193230677 shutdown_connections 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.755+0000 7fc69e46f740 1 -- 172.21.15.45:0/3193230677 wait complete. 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.756+0000 7fc69e46f740 1 -- start start 2024-09-10T10:04:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.756+0000 7fc69e46f740 1 Processor -- start 2024-09-10T10:04:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa91dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa92300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa92840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc69e46f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d6fa97810 con 0x563d6fa94e80 2024-09-10T10:04:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc69e46f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563d6f977290 con 0x563d6fa89cc0 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc69e46f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563d6fa97990 con 0x563d6fa8a5e0 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc69e46f740 10 client.0 _ll_register_callbacks cb 0x563d6f9a8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc699c76640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa92840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc699c76640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa92840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50834/0 (socket says 172.21.15.45:50834) 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.757+0000 7fc699c76640 1 -- 172.21.15.45:0/437161634 learned_addr learned my addr 172.21.15.45:0/437161634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699c76640 1 -- 172.21.15.45:0/437161634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 msgr2=0x563d6fa92300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699475640 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa91dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699c76640 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa92300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699c76640 1 -- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 msgr2=0x563d6fa91dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699c76640 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa91dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699c76640 1 -- 172.21.15.45:0/437161634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6f701210 con 0x563d6fa94e80 2024-09-10T10:04:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699475640 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa91dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc699c76640 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa92840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc68c017bd0 tx=0x7fc68c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc317349b47a21c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc67cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:04:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.758+0000 7fc67cff9640 1 -- 172.21.15.45:0/437161634 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00acb0 con 0x563d6fa94e80 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.759+0000 7fc67cff9640 1 -- 172.21.15.45:0/437161634 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fc68c021410 con 0x563d6fa94e80 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.759+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6840bcad0 con 0x563d6fa94e80 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.759+0000 7fc67cff9640 1 -- 172.21.15.45:0/437161634 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc68c00ae50 con 0x563d6fa94e80 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.759+0000 7fc69e46f740 10 client.15240 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.759+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563d6f7cd620 con 0x563d6fa94e80 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.760+0000 7fc67cff9640 1 -- 172.21.15.45:0/437161634 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6840bcad0 con 0x563d6fa94e80 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.760+0000 7fc67cff9640 1 -- 172.21.15.45:0/437161634 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc68c02a810 con 0x563d6fa94e80 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.760+0000 7fc69e46f740 10 client.15240 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.761+0000 7fc69e46f740 10 client.15240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.761+0000 7fc69e46f740 20 client.15240 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.761+0000 7fc69e46f740 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6faa6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.761+0000 7fc69e46f740 4 client.15240 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.761+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d6f9fba20 con 0x563d6faa4af0 2024-09-10T10:04:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.762+0000 7fc699475640 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6faa6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.763+0000 7fc699475640 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6faa6f80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beaac59a807dfeed server_cookie=eb53343399c734e8 in_seq=0 out_seq=0 2024-09-10T10:04:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.763+0000 7fc67cff9640 10 client.15240 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.767+0000 7fc67cff9640 1 -- 172.21.15.45:0/437161634 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d6f9fba20 con 0x563d6faa4af0 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.767+0000 7fc67cff9640 10 client.15240 handle_command_reply: tid=0 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.767+0000 7fc69e46f740 1 client.15240 shutdown 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.767+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 msgr2=0x563d6fa92840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.767+0000 7fc69e46f740 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa92840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc68c017bd0 tx=0x7fc68c004b50 comp rx=0 tx=0).stop 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 shutdown_connections 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6faa6f80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc67cff9640 0 client.15240 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa94e80 0x563d6fa92840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa92300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 --2- 172.21.15.45:0/437161634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa91dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 >> 172.21.15.45:0/437161634 conn(0x563d6fa86230 msgr2=0x563d6fa8bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 shutdown_connections 2024-09-10T10:04:20.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 1 -- 172.21.15.45:0/437161634 wait complete. 2024-09-10T10:04:20.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.768+0000 7fc69e46f740 20 client.15240 trim_cache size 0 max 16384 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 Processor -- start 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 -- start start 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa89cc0 0x563d6fa92670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa85170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa94e80 0x563d6faac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ad900 con 0x563d6fa89cc0 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941b01a0 con 0x563d6fa94e80 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc69e46f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941adf90 con 0x563d6fa8a5e0 2024-09-10T10:04:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc699475640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa89cc0 0x563d6fa92670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc699475640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa89cc0 0x563d6fa92670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50846/0 (socket says 172.21.15.45:50846) 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.785+0000 7fc699475640 1 -- 172.21.15.45:0/545170287 learned_addr learned my addr 172.21.15.45:0/545170287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc699c76640 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa94e80 0x563d6faac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc699475640 1 -- 172.21.15.45:0/545170287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 msgr2=0x563d6fa85170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc699475640 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa85170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc699475640 1 -- 172.21.15.45:0/545170287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa94e80 msgr2=0x563d6faac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc699475640 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa94e80 0x563d6faac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc699475640 1 -- 172.21.15.45:0/545170287 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6fa975b0 con 0x563d6fa89cc0 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc699475640 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa89cc0 0x563d6fa92670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc69000a160 tx=0x7fc69000ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db4ff98f51ef66a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc67effd640 1 -- 172.21.15.45:0/545170287 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690021070 con 0x563d6fa89cc0 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.786+0000 7fc67effd640 1 -- 172.21.15.45:0/545170287 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc69001c430 con 0x563d6fa89cc0 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.787+0000 7fc67effd640 1 -- 172.21.15.45:0/545170287 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690014670 con 0x563d6fa89cc0 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 -- 172.21.15.45:0/545170287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa89cc0 msgr2=0x563d6fa92670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa89cc0 0x563d6fa92670 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc69000a160 tx=0x7fc69000ace0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 -- 172.21.15.45:0/545170287 shutdown_connections 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa94e80 0x563d6faac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa8a5e0 0x563d6fa85170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 --2- 172.21.15.45:0/545170287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa89cc0 0x563d6fa92670 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 -- 172.21.15.45:0/545170287 >> 172.21.15.45:0/545170287 conn(0x563d6fa86230 msgr2=0x563d6fa84a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 -- 172.21.15.45:0/545170287 shutdown_connections 2024-09-10T10:04:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 -- 172.21.15.45:0/545170287 wait complete. 2024-09-10T10:04:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 -- start start 2024-09-10T10:04:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.788+0000 7fc69e46f740 1 Processor -- start 2024-09-10T10:04:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa9d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa8a5e0 0x563d6fa98720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc69e46f740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa94e80 0x563d6fa98cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc69e46f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941ae660 con 0x563d6fa8a5e0 2024-09-10T10:04:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc69e46f740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941afad0 con 0x563d6fa89cc0 2024-09-10T10:04:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc69e46f740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941aed30 con 0x563d6fa94e80 2024-09-10T10:04:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc69e46f740 10 client.0 _ll_register_callbacks cb 0x7fc68c017880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc698c74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa8a5e0 0x563d6fa98720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc698c74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa8a5e0 0x563d6fa98720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50858/0 (socket says 172.21.15.45:50858) 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.789+0000 7fc698c74640 1 -- 172.21.15.45:0/1677870270 learned_addr learned my addr 172.21.15.45:0/1677870270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.790+0000 7fc698c74640 1 -- 172.21.15.45:0/1677870270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa94e80 msgr2=0x563d6fa98cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.790+0000 7fc698c74640 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa94e80 0x563d6fa98cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.790+0000 7fc698c74640 1 -- 172.21.15.45:0/1677870270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 msgr2=0x563d6fa9d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.790+0000 7fc698c74640 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa9d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.790+0000 7fc698c74640 1 -- 172.21.15.45:0/1677870270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6fa994f0 con 0x563d6fa8a5e0 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.790+0000 7fc698c74640 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa8a5e0 0x563d6fa98720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b8830 tx=0x7fc684003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a1722c76187c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.790+0000 7fc67cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.791+0000 7fc67cff9640 1 -- 172.21.15.45:0/1677870270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684049070 con 0x563d6fa8a5e0 2024-09-10T10:04:20.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.791+0000 7fc67cff9640 1 -- 172.21.15.45:0/1677870270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fc6840bc630 con 0x563d6fa8a5e0 2024-09-10T10:04:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.791+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc69001c430 con 0x563d6fa8a5e0 2024-09-10T10:04:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.791+0000 7fc67cff9640 1 -- 172.21.15.45:0/1677870270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc6840bc1a0 con 0x563d6fa8a5e0 2024-09-10T10:04:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 10 client.15246 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563d6fb0f990 con 0x563d6fa8a5e0 2024-09-10T10:04:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc67cff9640 1 -- 172.21.15.45:0/1677870270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6840bcad0 con 0x563d6fa8a5e0 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc67cff9640 1 -- 172.21.15.45:0/1677870270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fc6840f3170 con 0x563d6fa8a5e0 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 10 client.15246 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 10 client.15246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 20 client.15246 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6fb11f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 4 client.15246 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.792+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc69001c430 con 0x563d6faa4af0 2024-09-10T10:04:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.793+0000 7fc699475640 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6fb11f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.797+0000 7fc67cff9640 10 client.15246 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:20.797+0000 7fc699475640 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6fb11f30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514dc707cd55fa4f server_cookie=1adddeea4d34cc7f in_seq=0 out_seq=0 2024-09-10T10:04:21.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:21 smithi045 ceph-mon[31114]: pgmap v37: 97 pgs: 97 active+clean; 395 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-10T10:04:21.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:21 smithi047 ceph-mon[35244]: pgmap v37: 97 pgs: 97 active+clean; 395 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-10T10:04:21.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:21 smithi162 ceph-mon[34963]: pgmap v37: 97 pgs: 97 active+clean; 395 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-10T10:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc67cff9640 1 -- 172.21.15.45:0/1677870270 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc69001c430 con 0x563d6faa4af0 2024-09-10T10:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc67cff9640 10 client.15246 handle_command_reply: tid=0 2024-09-10T10:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 client.15246 shutdown 2024-09-10T10:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa8a5e0 msgr2=0x563d6fa98720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa8a5e0 0x563d6fa98720 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6840b8830 tx=0x7fc684003eb0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 shutdown_connections 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563d6faa4af0 0x563d6fb11f30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563d6fa94e80 0x563d6fa98cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d6fa8a5e0 0x563d6fa98720 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc67cff9640 0 client.15246 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 --2- 172.21.15.45:0/1677870270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563d6fa89cc0 0x563d6fa9d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.655+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 >> 172.21.15.45:0/1677870270 conn(0x563d6fa86230 msgr2=0x563d6fa8b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.656+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 shutdown_connections 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.656+0000 7fc69e46f740 1 -- 172.21.15.45:0/1677870270 wait complete. 2024-09-10T10:04:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.656+0000 7fc69e46f740 20 client.15246 trim_cache size 0 max 16384 2024-09-10T10:04:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:04:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:04:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:04:21.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:04:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc6800a2f10 msgr2=0x7fc6800a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc6800a2f10 0x7fc6800a53d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e10 tx=0x7fc68c004a40 comp rx=0 tx=0).stop 2024-09-10T10:04:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6941044a0 msgr2=0x7fc69419aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6941044a0 0x7fc69419aa60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc684006010 tx=0x7fc684032b40 comp rx=0 tx=0).stop 2024-09-10T10:04:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 shutdown_connections 2024-09-10T10:04:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc694109ba0 0x7fc69419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc6941044a0 0x7fc69419aa60 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc684006010 tx=0x7fc684032b40 comp rx=0 tx=0).stop 2024-09-10T10:04:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc6800a2f10 0x7fc6800a53d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 --2- 172.21.15.45:0/3532465059 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc694103b00 0x7fc69419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.657+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 >> 172.21.15.45:0/3532465059 conn(0x7fc6940ff7c0 msgr2=0x7fc6940fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.658+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 shutdown_connections 2024-09-10T10:04:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:21.658+0000 7fc69a477640 1 -- 172.21.15.45:0/3532465059 wait complete. 2024-09-10T10:04:23.220 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:23 smithi045 ceph-mon[31114]: pgmap v38: 97 pgs: 97 active+clean; 395 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-10T10:04:23.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:23 smithi047 ceph-mon[35244]: pgmap v38: 97 pgs: 97 active+clean; 395 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-10T10:04:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:23 smithi162 ceph-mon[34963]: pgmap v38: 97 pgs: 97 active+clean; 395 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 33 MiB/s wr, 34 op/s 2024-09-10T10:04:25.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:25 smithi162 ceph-mon[34963]: pgmap v39: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 400 MiB/s wr, 330 op/s 2024-09-10T10:04:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:25 smithi045 ceph-mon[31114]: pgmap v39: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 400 MiB/s wr, 330 op/s 2024-09-10T10:04:25.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:25 smithi047 ceph-mon[35244]: pgmap v39: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 821 KiB/s rd, 400 MiB/s wr, 330 op/s 2024-09-10T10:04:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:27 smithi162 ceph-mon[34963]: pgmap v40: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 400 MiB/s wr, 329 op/s 2024-09-10T10:04:27.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:27 smithi045 ceph-mon[31114]: pgmap v40: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 400 MiB/s wr, 329 op/s 2024-09-10T10:04:27.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:27 smithi047 ceph-mon[35244]: pgmap v40: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 400 MiB/s wr, 329 op/s 2024-09-10T10:04:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:29 smithi045 ceph-mon[31114]: pgmap v41: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 752 MiB/s wr, 886 op/s 2024-09-10T10:04:29.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:29 smithi047 ceph-mon[35244]: pgmap v41: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 752 MiB/s wr, 886 op/s 2024-09-10T10:04:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:29 smithi162 ceph-mon[34963]: pgmap v41: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 752 MiB/s wr, 886 op/s 2024-09-10T10:04:31.701 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:31 smithi162 ceph-mon[34963]: pgmap v42: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 725 MiB/s wr, 854 op/s 2024-09-10T10:04:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:31 smithi045 ceph-mon[31114]: pgmap v42: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 725 MiB/s wr, 854 op/s 2024-09-10T10:04:31.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:31 smithi047 ceph-mon[35244]: pgmap v42: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 725 MiB/s wr, 854 op/s 2024-09-10T10:04:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:33 smithi162 ceph-mon[34963]: pgmap v43: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 719 MiB/s wr, 853 op/s 2024-09-10T10:04:33.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:33 smithi045 ceph-mon[31114]: pgmap v43: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 719 MiB/s wr, 853 op/s 2024-09-10T10:04:33.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:33 smithi047 ceph-mon[35244]: pgmap v43: 97 pgs: 97 active+clean; 8.4 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 719 MiB/s wr, 853 op/s 2024-09-10T10:04:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4214430520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1009d0 msgr2=0x7f179c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4214430520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1009d0 0x7f179c100e30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1788009930 tx=0x7f178802a480 comp rx=0 tx=0).stop 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4214430520 shutdown_connections 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4214430520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c101370 0x7f179c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4214430520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1009d0 0x7f179c100e30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4214430520 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f179c1069e0 0x7f179c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4214430520 >> 172.21.15.45:0/4214430520 conn(0x7f179c0fc6f0 msgr2=0x7f179c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4214430520 shutdown_connections 2024-09-10T10:04:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.886+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4214430520 wait complete. 2024-09-10T10:04:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.888+0000 7f17a2ca5640 1 Processor -- start 2024-09-10T10:04:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.888+0000 7f17a2ca5640 1 -- start start 2024-09-10T10:04:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.888+0000 7f17a2ca5640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c1009d0 0x7f179c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a2ca5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f179c101370 0x7f179c1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a2ca5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1069e0 0x7f179c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a2ca5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f179c110f10 con 0x7f179c1069e0 2024-09-10T10:04:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a2ca5640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f179c110d90 con 0x7f179c1009d0 2024-09-10T10:04:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a2ca5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f179c111090 con 0x7f179c101370 2024-09-10T10:04:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a0a41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c1009d0 0x7f179c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a1242640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1069e0 0x7f179c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a1242640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1069e0 0x7f179c19bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48448/0 (socket says 172.21.15.45:48448) 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.889+0000 7f17a1242640 1 -- 172.21.15.45:0/4055176807 learned_addr learned my addr 172.21.15.45:0/4055176807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.890+0000 7f17a1242640 1 -- 172.21.15.45:0/4055176807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f179c101370 msgr2=0x7f179c1979a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.890+0000 7f17a1242640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f179c101370 0x7f179c1979a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.890+0000 7f17a1242640 1 -- 172.21.15.45:0/4055176807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c1009d0 msgr2=0x7f179c197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.890+0000 7f17a1242640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c1009d0 0x7f179c197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.890+0000 7f17a1242640 1 -- 172.21.15.45:0/4055176807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c19c420 con 0x7f179c1069e0 2024-09-10T10:04:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.890+0000 7f17a1242640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1069e0 0x7f179c19bd20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f179400b990 tx=0x7f179400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.892+0000 7f1799ffb640 1 -- 172.21.15.45:0/4055176807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794019070 con 0x7f179c1069e0 2024-09-10T10:04:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.892+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c19c710 con 0x7f179c1069e0 2024-09-10T10:04:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.893+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c06a280 con 0x7f179c1069e0 2024-09-10T10:04:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.893+0000 7f1799ffb640 1 -- 172.21.15.45:0/4055176807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179400cba0 con 0x7f179c1069e0 2024-09-10T10:04:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.895+0000 7f1799ffb640 1 -- 172.21.15.45:0/4055176807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940076d0 con 0x7f179c1069e0 2024-09-10T10:04:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.904+0000 7f17a5a44740 1 Processor -- start 2024-09-10T10:04:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.905+0000 7f1799ffb640 1 -- 172.21.15.45:0/4055176807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f179401c070 con 0x7f179c1069e0 2024-09-10T10:04:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.906+0000 7f17a5a44740 1 -- start start 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.906+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.906+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.906+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58e20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.907+0000 7f17a5a44740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ded58392f0 con 0x55ded58d5410 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.907+0000 7f17a5a44740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ded57e1270 con 0x55ded58d4af0 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.907+0000 7f17a5a44740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ded58352b0 con 0x55ded58dfcb0 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.908+0000 7f179bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.908+0000 7f179bfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:34062/0 (socket says 172.21.15.45:34062) 2024-09-10T10:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.908+0000 7f179bfff640 1 -- 172.21.15.45:0/618208938 learned_addr learned my addr 172.21.15.45:0/618208938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.908+0000 7f179bfff640 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58e20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.909+0000 7f17a0a41640 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.910+0000 7f179bfff640 1 -- 172.21.15.45:0/618208938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 msgr2=0x55ded58e20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.910+0000 7f179bfff640 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58e20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.910+0000 7f179bfff640 1 -- 172.21.15.45:0/618208938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 msgr2=0x55ded58df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.910+0000 7f179bfff640 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.910+0000 7f179bfff640 1 -- 172.21.15.45:0/618208938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded57e0340 con 0x55ded58d4af0 2024-09-10T10:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.910+0000 7f1799ffb640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2eb0 0x7f178c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.911+0000 7f1799ffb640 1 -- 172.21.15.45:0/4055176807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f17940ceba0 con 0x7f179c1069e0 2024-09-10T10:04:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.911+0000 7f179bfff640 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1788033000 tx=0x7f17880315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a1c413761bbc92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.912+0000 7f1786ffd640 1 -- 172.21.15.45:0/618208938 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788035070 con 0x55ded58d4af0 2024-09-10T10:04:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.912+0000 7f17a0a41640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2eb0 0x7f178c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.912+0000 7f1786ffd640 1 -- 172.21.15.45:0/618208938 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788004230 con 0x55ded58d4af0 2024-09-10T10:04:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.912+0000 7f1786ffd640 1 -- 172.21.15.45:0/618208938 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178803d650 con 0x55ded58d4af0 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a5a44740 1 -- 172.21.15.45:0/618208938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 msgr2=0x55ded58d4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a5a44740 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d4ed0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1788033000 tx=0x7f17880315b0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a0a41640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2eb0 0x7f178c0a5370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1790006fd0 tx=0x7f1790008040 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a5a44740 1 -- 172.21.15.45:0/618208938 shutdown_connections 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a5a44740 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58e20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a5a44740 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a5a44740 1 --2- 172.21.15.45:0/618208938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d4ed0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.913+0000 7f17a5a44740 1 -- 172.21.15.45:0/618208938 >> 172.21.15.45:0/618208938 conn(0x55ded58d1060 msgr2=0x55ded58d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.914+0000 7f17a5a44740 1 -- 172.21.15.45:0/618208938 shutdown_connections 2024-09-10T10:04:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.914+0000 7f17a5a44740 1 -- 172.21.15.45:0/618208938 wait complete. 2024-09-10T10:04:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.915+0000 7f17a5a44740 1 -- start start 2024-09-10T10:04:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.915+0000 7f17a5a44740 1 Processor -- start 2024-09-10T10:04:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.916+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.916+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a5a44740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ded58e28d0 con 0x55ded58d5410 2024-09-10T10:04:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a5a44740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ded57c2290 con 0x55ded58d4af0 2024-09-10T10:04:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a0a41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a0a41640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58dcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:34070/0 (socket says 172.21.15.45:34070) 2024-09-10T10:04:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a0a41640 1 -- 172.21.15.45:0/580096920 learned_addr learned my addr 172.21.15.45:0/580096920 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ded58e2a50 con 0x55ded58dfcb0 2024-09-10T10:04:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f17a5a44740 10 client.0 _ll_register_callbacks cb 0x55ded57f3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.917+0000 7f179bfff640 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f17a0a41640 1 -- 172.21.15.45:0/580096920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 msgr2=0x55ded58dd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f17a0a41640 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f17a0a41640 1 -- 172.21.15.45:0/580096920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 msgr2=0x55ded58dd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f17a0a41640 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f17a0a41640 1 -- 172.21.15.45:0/580096920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded554c210 con 0x55ded58d4af0 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f179bfff640 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f17a0a41640 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58dcb10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17900167d0 tx=0x7f17900046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738c20ce9b4ce5f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.918+0000 7f177bfff640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:04:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.920+0000 7f177bfff640 1 -- 172.21.15.45:0/580096920 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179001c050 con 0x55ded58d4af0 2024-09-10T10:04:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.921+0000 7f177bfff640 1 -- 172.21.15.45:0/580096920 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f1790014930 con 0x55ded58d4af0 2024-09-10T10:04:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.921+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1788004230 con 0x55ded58d4af0 2024-09-10T10:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.923+0000 7f177bfff640 1 -- 172.21.15.45:0/580096920 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1790014b30 con 0x55ded58d4af0 2024-09-10T10:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.924+0000 7f17a5a44740 10 client.24977 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.924+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ded5618620 con 0x55ded58d4af0 2024-09-10T10:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.925+0000 7f177bfff640 1 -- 172.21.15.45:0/580096920 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788004230 con 0x55ded58d4af0 2024-09-10T10:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.926+0000 7f177bfff640 1 -- 172.21.15.45:0/580096920 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1790006510 con 0x55ded58d4af0 2024-09-10T10:04:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.926+0000 7f17a5a44740 10 client.24977 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.926+0000 7f17a5a44740 10 client.24977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.926+0000 7f17a5a44740 20 client.24977 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.926+0000 7f17a5a44740 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded58f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.926+0000 7f17a5a44740 4 client.24977 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.926+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ded5846a20 con 0x55ded58efe20 2024-09-10T10:04:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.927+0000 7f179bfff640 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded58f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.930+0000 7f177bfff640 10 client.24977 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.930+0000 7f179bfff640 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded58f22e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379516cfdbc5c470 server_cookie=a621cddad21f0842 in_seq=0 out_seq=0 2024-09-10T10:04:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.934+0000 7f177bfff640 1 -- 172.21.15.45:0/580096920 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ded5846a20 con 0x55ded58efe20 2024-09-10T10:04:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.934+0000 7f177bfff640 10 client.24977 handle_command_reply: tid=0 2024-09-10T10:04:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.934+0000 7f17a5a44740 1 client.24977 shutdown 2024-09-10T10:04:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 msgr2=0x55ded58dcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58dcb10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17900167d0 tx=0x7f17900046e0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 shutdown_connections 2024-09-10T10:04:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded58f22e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f177bfff640 0 client.24977 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 --2- 172.21.15.45:0/580096920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58dcb10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.935+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 >> 172.21.15.45:0/580096920 conn(0x55ded58d1060 msgr2=0x55ded58d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.936+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 shutdown_connections 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.936+0000 7f17a5a44740 1 -- 172.21.15.45:0/580096920 wait complete. 2024-09-10T10:04:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.936+0000 7f17a5a44740 20 client.24977 trim_cache size 0 max 16384 2024-09-10T10:04:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.962+0000 7f17a5a44740 1 Processor -- start 2024-09-10T10:04:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.962+0000 7f17a5a44740 1 -- start start 2024-09-10T10:04:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.962+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.962+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48474/0 (socket says 172.21.15.45:48474) 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 -- 172.21.15.45:0/650912412 learned_addr learned my addr 172.21.15.45:0/650912412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f17a5a44740 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f178000c700 con 0x55ded58d5410 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f178000efa0 con 0x55ded58d4af0 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f178000cd90 con 0x55ded58dfcb0 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f17a0a41640 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 -- 172.21.15.45:0/650912412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 msgr2=0x55ded58f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 -- 172.21.15.45:0/650912412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 msgr2=0x55ded58d75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.963+0000 7f179bfff640 1 -- 172.21.15.45:0/650912412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded58d0e90 con 0x55ded58d5410 2024-09-10T10:04:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.964+0000 7f179bfff640 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f178802a9a0 tx=0x7f1788003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f964431a65997b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.964+0000 7f17867fc640 1 -- 172.21.15.45:0/650912412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788054030 con 0x55ded58d5410 2024-09-10T10:04:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.964+0000 7f17867fc640 1 -- 172.21.15.45:0/650912412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788004460 con 0x55ded58d5410 2024-09-10T10:04:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.964+0000 7f17867fc640 1 -- 172.21.15.45:0/650912412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788035040 con 0x55ded58d5410 2024-09-10T10:04:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.965+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 msgr2=0x55ded58dd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.965+0000 7f17a5a44740 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f178802a9a0 tx=0x7f1788003c50 comp rx=0 tx=0).stop 2024-09-10T10:04:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 shutdown_connections 2024-09-10T10:04:34.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58dd3c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 --2- 172.21.15.45:0/650912412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58d75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 >> 172.21.15.45:0/650912412 conn(0x55ded58d1060 msgr2=0x55ded58deb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 shutdown_connections 2024-09-10T10:04:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 -- 172.21.15.45:0/650912412 wait complete. 2024-09-10T10:04:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.966+0000 7f17a5a44740 1 -- start start 2024-09-10T10:04:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.967+0000 7f17a5a44740 1 Processor -- start 2024-09-10T10:04:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.967+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.967+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f17a5a44740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58d9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f17a5a44740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f178000d460 con 0x55ded58d5410 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f17a5a44740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f178000e8d0 con 0x55ded58d4af0 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f17a5a44740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f178000db30 con 0x55ded58dfcb0 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f179bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f179bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58d9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48484/0 (socket says 172.21.15.45:48484) 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f179bfff640 1 -- 172.21.15.45:0/2746192581 learned_addr learned my addr 172.21.15.45:0/2746192581 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f17a5a44740 10 client.0 _ll_register_callbacks cb 0x7f179000b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f179bfff640 1 -- 172.21.15.45:0/2746192581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 msgr2=0x55ded58d9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f179bfff640 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58d9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f179bfff640 1 -- 172.21.15.45:0/2746192581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 msgr2=0x55ded58e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.968+0000 7f179bfff640 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.969+0000 7f179bfff640 1 -- 172.21.15.45:0/2746192581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded58da610 con 0x55ded58d5410 2024-09-10T10:04:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.970+0000 7f179bfff640 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58d9880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1788031ae0 tx=0x7f17880041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f461c67eae9403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.970+0000 7f177bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:04:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.970+0000 7f177bfff640 1 -- 172.21.15.45:0/2746192581 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788054040 con 0x55ded58d5410 2024-09-10T10:04:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.970+0000 7f177bfff640 1 -- 172.21.15.45:0/2746192581 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f17880037f0 con 0x55ded58d5410 2024-09-10T10:04:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.970+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1788004460 con 0x55ded58d5410 2024-09-10T10:04:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.971+0000 7f177bfff640 1 -- 172.21.15.45:0/2746192581 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f1788035040 con 0x55ded58d5410 2024-09-10T10:04:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.971+0000 7f17a5a44740 10 client.15261 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.971+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ded5963450 con 0x55ded58d5410 2024-09-10T10:04:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.972+0000 7f177bfff640 1 -- 172.21.15.45:0/2746192581 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17880039f0 con 0x55ded58d5410 2024-09-10T10:04:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.972+0000 7f177bfff640 1 -- 172.21.15.45:0/2746192581 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f1788061030 con 0x55ded58d5410 2024-09-10T10:04:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.973+0000 7f17a5a44740 10 client.15261 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.973+0000 7f17a5a44740 10 client.15261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.973+0000 7f17a5a44740 20 client.15261 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.973+0000 7f17a5a44740 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded5965690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.973+0000 7f17a5a44740 4 client.15261 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:04:34.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.973+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1788004460 con 0x55ded58efe20 2024-09-10T10:04:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.974+0000 7f17a0a41640 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded5965690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.974+0000 7f17a0a41640 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded5965690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=448a2423006da805 server_cookie=a08e3ea8b3310816 in_seq=0 out_seq=0 2024-09-10T10:04:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:34.975+0000 7f177bfff640 10 client.15261 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f177bfff640 1 -- 172.21.15.45:0/2746192581 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1788004460 con 0x55ded58efe20 2024-09-10T10:04:35.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f177bfff640 10 client.15261 handle_command_reply: tid=0 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 client.15261 shutdown 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 msgr2=0x55ded58d9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58d9880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1788031ae0 tx=0x7f17880041a0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 shutdown_connections 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ded58efe20 0x55ded5965690 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ded58dfcb0 0x55ded58d9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ded58d5410 0x55ded58d9880 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 --2- 172.21.15.45:0/2746192581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ded58d4af0 0x55ded58e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 >> 172.21.15.45:0/2746192581 conn(0x55ded58d1060 msgr2=0x55ded58cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.655+0000 7f177bfff640 0 client.15261 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.656+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 shutdown_connections 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.656+0000 7f17a5a44740 1 -- 172.21.15.45:0/2746192581 wait complete. 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.656+0000 7f17a5a44740 20 client.15261 trim_cache size 0 max 16384 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:04:35.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:04:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.663+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2eb0 msgr2=0x7f178c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.663+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2eb0 0x7f178c0a5370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1790006fd0 tx=0x7f1790008040 comp rx=0 tx=0).stop 2024-09-10T10:04:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.663+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1069e0 msgr2=0x7f179c19bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.663+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1069e0 0x7f179c19bd20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f179400b990 tx=0x7f179400be50 comp rx=0 tx=0).stop 2024-09-10T10:04:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.666+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 shutdown_connections 2024-09-10T10:04:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.666+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c1069e0 0x7f179c19bd20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.666+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f179c101370 0x7f179c1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.666+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2eb0 0x7f178c0a5370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.666+0000 7f17a2ca5640 1 --2- 172.21.15.45:0/4055176807 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f179c1009d0 0x7f179c197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.666+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 >> 172.21.15.45:0/4055176807 conn(0x7f179c0fc6f0 msgr2=0x7f179c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.671+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 shutdown_connections 2024-09-10T10:04:35.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:35.671+0000 7f17a2ca5640 1 -- 172.21.15.45:0/4055176807 wait complete. 2024-09-10T10:04:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:35 smithi162 ceph-mon[34963]: pgmap v44: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 GiB/s wr, 1.65k op/s 2024-09-10T10:04:35.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:35 smithi045 ceph-mon[31114]: pgmap v44: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 GiB/s wr, 1.65k op/s 2024-09-10T10:04:35.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:35 smithi047 ceph-mon[35244]: pgmap v44: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 GiB/s wr, 1.65k op/s 2024-09-10T10:04:37.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:36 smithi162 ceph-mon[34963]: pgmap v45: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 724 MiB/s wr, 1.36k op/s 2024-09-10T10:04:37.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:36 smithi045 ceph-mon[31114]: pgmap v45: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 724 MiB/s wr, 1.36k op/s 2024-09-10T10:04:37.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:36 smithi047 ceph-mon[35244]: pgmap v45: 97 pgs: 97 active+clean; 10 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 724 MiB/s wr, 1.36k op/s 2024-09-10T10:04:39.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:38 smithi162 ceph-mon[34963]: pgmap v46: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 999 MiB/s wr, 2.30k op/s 2024-09-10T10:04:39.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:38 smithi045 ceph-mon[31114]: pgmap v46: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 999 MiB/s wr, 2.30k op/s 2024-09-10T10:04:39.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:38 smithi047 ceph-mon[35244]: pgmap v46: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 999 MiB/s wr, 2.30k op/s 2024-09-10T10:04:41.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:40 smithi162 ceph-mon[34963]: pgmap v47: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 718 MiB/s wr, 1.96k op/s 2024-09-10T10:04:41.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:40 smithi045 ceph-mon[31114]: pgmap v47: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 718 MiB/s wr, 1.96k op/s 2024-09-10T10:04:41.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:40 smithi047 ceph-mon[35244]: pgmap v47: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 718 MiB/s wr, 1.96k op/s 2024-09-10T10:04:43.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:42 smithi162 ceph-mon[34963]: pgmap v48: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 718 MiB/s wr, 1.96k op/s 2024-09-10T10:04:43.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:42 smithi045 ceph-mon[31114]: pgmap v48: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 718 MiB/s wr, 1.96k op/s 2024-09-10T10:04:43.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:42 smithi047 ceph-mon[35244]: pgmap v48: 97 pgs: 97 active+clean; 10 GiB data, 37 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 718 MiB/s wr, 1.96k op/s 2024-09-10T10:04:45.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:44 smithi162 ceph-mon[34963]: pgmap v49: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 GiB/s wr, 2.89k op/s 2024-09-10T10:04:45.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:44 smithi162 ceph-mon[34963]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-10T10:04:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:44 smithi045 ceph-mon[31114]: pgmap v49: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 GiB/s wr, 2.89k op/s 2024-09-10T10:04:45.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:44 smithi045 ceph-mon[31114]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-10T10:04:45.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:44 smithi047 ceph-mon[35244]: pgmap v49: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 GiB/s wr, 2.89k op/s 2024-09-10T10:04:45.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:44 smithi047 ceph-mon[35244]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-10T10:04:47.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:46 smithi162 ceph-mon[34963]: pgmap v50: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 721 MiB/s wr, 2.09k op/s 2024-09-10T10:04:47.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:46 smithi045 ceph-mon[31114]: pgmap v50: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 721 MiB/s wr, 2.09k op/s 2024-09-10T10:04:47.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:46 smithi047 ceph-mon[35244]: pgmap v50: 97 pgs: 97 active+clean; 10 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 721 MiB/s wr, 2.09k op/s 2024-09-10T10:04:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.898+0000 7f8f4d991640 1 -- 172.21.15.45:0/4033025638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f481091f0 msgr2=0x7f8f48110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.898+0000 7f8f4d991640 1 --2- 172.21.15.45:0/4033025638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f481091f0 0x7f8f48110250 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009900 tx=0x7f8f3802a440 comp rx=0 tx=0).stop 2024-09-10T10:04:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.900+0000 7f8f4d991640 1 -- 172.21.15.45:0/4033025638 shutdown_connections 2024-09-10T10:04:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.900+0000 7f8f4d991640 1 --2- 172.21.15.45:0/4033025638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 0x7f8f4811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.900+0000 7f8f4d991640 1 --2- 172.21.15.45:0/4033025638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481095d0 0x7f8f48109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.900+0000 7f8f4d991640 1 --2- 172.21.15.45:0/4033025638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f481091f0 0x7f8f48110250 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.900+0000 7f8f4d991640 1 -- 172.21.15.45:0/4033025638 >> 172.21.15.45:0/4033025638 conn(0x7f8f4806ce00 msgr2=0x7f8f4806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.900+0000 7f8f4d991640 1 -- 172.21.15.45:0/4033025638 shutdown_connections 2024-09-10T10:04:48.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.900+0000 7f8f4d991640 1 -- 172.21.15.45:0/4033025638 wait complete. 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.901+0000 7f8f4d991640 1 Processor -- start 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.902+0000 7f8f4d991640 1 -- start start 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.902+0000 7f8f4d991640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f481091f0 0x7f8f4811a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.902+0000 7f8f4d991640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481095d0 0x7f8f48114960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.903+0000 7f8f4d991640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 0x7f8f48114ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.903+0000 7f8f4d991640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48111370 con 0x7f8f481091f0 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.903+0000 7f8f4d991640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481111f0 con 0x7f8f481095d0 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.903+0000 7f8f4d991640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481114f0 con 0x7f8f4811b140 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.903+0000 7f8f477fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 0x7f8f48114ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.903+0000 7f8f477fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 0x7f8f48114ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:55040/0 (socket says 172.21.15.45:55040) 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.903+0000 7f8f477fe640 1 -- 172.21.15.45:0/1287197741 learned_addr learned my addr 172.21.15.45:0/1287197741 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:48.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.904+0000 7f8f477fe640 1 -- 172.21.15.45:0/1287197741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481095d0 msgr2=0x7f8f48114960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:04:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.904+0000 7f8f477fe640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481095d0 0x7f8f48114960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.904+0000 7f8f477fe640 1 -- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f481091f0 msgr2=0x7f8f4811a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.904+0000 7f8f477fe640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f481091f0 0x7f8f4811a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.904+0000 7f8f477fe640 1 -- 172.21.15.45:0/1287197741 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f48115670 con 0x7f8f4811b140 2024-09-10T10:04:48.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.905+0000 7f8f477fe640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 0x7f8f48114ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b990 tx=0x7f8f4000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.912+0000 7f8f4c98f640 1 -- 172.21.15.45:0/1287197741 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40019070 con 0x7f8f4811b140 2024-09-10T10:04:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.912+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4807a450 con 0x7f8f4811b140 2024-09-10T10:04:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.912+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4807a940 con 0x7f8f4811b140 2024-09-10T10:04:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.913+0000 7f8f4c98f640 1 -- 172.21.15.45:0/1287197741 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f4000cba0 con 0x7f8f4811b140 2024-09-10T10:04:48.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.913+0000 7f8f4c98f640 1 -- 172.21.15.45:0/1287197741 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400076e0 con 0x7f8f4811b140 2024-09-10T10:04:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.918+0000 7f8f4c98f640 1 -- 172.21.15.45:0/1287197741 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8f4001c070 con 0x7f8f4811b140 2024-09-10T10:04:48.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.919+0000 7f8f4c98f640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8f300a2fd0 0x7f8f300a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.920+0000 7f8f4c98f640 1 -- 172.21.15.45:0/1287197741 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f8f400cf980 con 0x7f8f4811b140 2024-09-10T10:04:48.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.922+0000 7f8f46ffd640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8f300a2fd0 0x7f8f300a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.926+0000 7f8f46ffd640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8f300a2fd0 0x7f8f300a5490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005c20 tx=0x7f8f38000bc0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.928+0000 7f8f5080b740 1 Processor -- start 2024-09-10T10:04:48.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.928+0000 7f8f5080b740 1 -- start start 2024-09-10T10:04:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.930+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed89ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.930+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed8a410 0x55a2bed94770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.930+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed94cb0 0x55a2bed970a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.930+0000 7f8f5080b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a2becee2f0 con 0x55a2bed89af0 2024-09-10T10:04:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.930+0000 7f8f5080b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a2bec96270 con 0x55a2bed94cb0 2024-09-10T10:04:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.931+0000 7f8f5080b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a2becea2b0 con 0x55a2bed8a410 2024-09-10T10:04:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.931+0000 7f8f467fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed8a410 0x55a2bed94770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.931+0000 7f8f467fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed8a410 0x55a2bed94770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:55052/0 (socket says 172.21.15.45:55052) 2024-09-10T10:04:48.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.931+0000 7f8f467fc640 1 -- 172.21.15.45:0/1873142465 learned_addr learned my addr 172.21.15.45:0/1873142465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.932+0000 7f8f467fc640 1 -- 172.21.15.45:0/1873142465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed94cb0 msgr2=0x55a2bed970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.932+0000 7f8f467fc640 1 --2- 172.21.15.45:0/1873142465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed94cb0 0x55a2bed970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.932+0000 7f8f467fc640 1 -- 172.21.15.45:0/1873142465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 msgr2=0x55a2bed89ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.932+0000 7f8f467fc640 1 --2- 172.21.15.45:0/1873142465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed89ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.932+0000 7f8f467fc640 1 -- 172.21.15.45:0/1873142465 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bec95340 con 0x55a2bed8a410 2024-09-10T10:04:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.934+0000 7f8f467fc640 1 --2- 172.21.15.45:0/1873142465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed8a410 0x55a2bed94770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0096f0 tx=0x7f8f3c009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecc99b839d1e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.944+0000 7f8f2dffb640 1 -- 172.21.15.45:0/1873142465 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c015020 con 0x55a2bed8a410 2024-09-10T10:04:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.944+0000 7f8f2dffb640 1 -- 172.21.15.45:0/1873142465 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c00bdb0 con 0x55a2bed8a410 2024-09-10T10:04:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.944+0000 7f8f2dffb640 1 -- 172.21.15.45:0/1873142465 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0103e0 con 0x55a2bed8a410 2024-09-10T10:04:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.946+0000 7f8f5080b740 1 -- 172.21.15.45:0/1873142465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed8a410 msgr2=0x55a2bed94770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.946+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1873142465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed8a410 0x55a2bed94770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0096f0 tx=0x7f8f3c009bb0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.947+0000 7f8f5080b740 1 -- 172.21.15.45:0/1873142465 shutdown_connections 2024-09-10T10:04:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.947+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1873142465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed94cb0 0x55a2bed970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.947+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1873142465 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed8a410 0x55a2bed94770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.947+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1873142465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed89ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.947+0000 7f8f5080b740 1 -- 172.21.15.45:0/1873142465 >> 172.21.15.45:0/1873142465 conn(0x55a2bed86060 msgr2=0x55a2bed86440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.947+0000 7f8f5080b740 1 -- 172.21.15.45:0/1873142465 shutdown_connections 2024-09-10T10:04:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.947+0000 7f8f5080b740 1 -- 172.21.15.45:0/1873142465 wait complete. 2024-09-10T10:04:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.948+0000 7f8f5080b740 1 -- start start 2024-09-10T10:04:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.948+0000 7f8f5080b740 1 Processor -- start 2024-09-10T10:04:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.949+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed91bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.949+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed92100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.949+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bed92640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.950+0000 7f8f5080b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a2bed97880 con 0x55a2bed89af0 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.950+0000 7f8f5080b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a2bec77290 con 0x55a2bed8a410 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.950+0000 7f8f5080b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a2bed97a00 con 0x55a2bed94cb0 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.950+0000 7f8f5080b740 10 client.0 _ll_register_callbacks cb 0x55a2beca8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.950+0000 7f8f477fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bed92640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.950+0000 7f8f477fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bed92640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:55058/0 (socket says 172.21.15.45:55058) 2024-09-10T10:04:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.950+0000 7f8f477fe640 1 -- 172.21.15.45:0/2732271591 learned_addr learned my addr 172.21.15.45:0/2732271591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.951+0000 7f8f467fc640 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed92100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.951+0000 7f8f477fe640 1 -- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 msgr2=0x55a2bed92100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.951+0000 7f8f477fe640 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed92100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.951+0000 7f8f477fe640 1 -- 172.21.15.45:0/2732271591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 msgr2=0x55a2bed91bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.951+0000 7f8f477fe640 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed91bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.951+0000 7f8f477fe640 1 -- 172.21.15.45:0/2732271591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bea01210 con 0x55a2bed94cb0 2024-09-10T10:04:48.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.952+0000 7f8f477fe640 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bed92640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f400983e0 tx=0x7f8f4009c0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d05ed8fcc5cf38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.956+0000 7f8f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:04:48.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.956+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2732271591 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400985f0 con 0x55a2bed94cb0 2024-09-10T10:04:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.956+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3c00bdb0 con 0x55a2bed94cb0 2024-09-10T10:04:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.956+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2732271591 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f8f400cd320 con 0x55a2bed94cb0 2024-09-10T10:04:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.957+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2732271591 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8f40098790 con 0x55a2bed94cb0 2024-09-10T10:04:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.957+0000 7f8f5080b740 10 client.24898 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:48.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.957+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a2beacd620 con 0x55a2bed94cb0 2024-09-10T10:04:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.957+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2732271591 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c00bdb0 con 0x55a2bed94cb0 2024-09-10T10:04:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.959+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2732271591 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8f4009b3d0 con 0x55a2bed94cb0 2024-09-10T10:04:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.959+0000 7f8f5080b740 10 client.24898 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.959+0000 7f8f5080b740 10 client.24898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.959+0000 7f8f5080b740 20 client.24898 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.959+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2beda5180 0x55a2beda7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.959+0000 7f8f5080b740 4 client.24898 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.959+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2becfba20 con 0x55a2beda5180 2024-09-10T10:04:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.963+0000 7f8f46ffd640 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2beda5180 0x55a2beda7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.967+0000 7f8f26ffd640 10 client.24898 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:48.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.967+0000 7f8f46ffd640 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2beda5180 0x55a2beda7600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d0bc08eccc31 server_cookie=ce8ef653dd6394a6 in_seq=0 out_seq=0 2024-09-10T10:04:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.973+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2732271591 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2becfba20 con 0x55a2beda5180 2024-09-10T10:04:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.973+0000 7f8f26ffd640 10 client.24898 handle_command_reply: tid=0 2024-09-10T10:04:48.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.973+0000 7f8f5080b740 1 client.24898 shutdown 2024-09-10T10:04:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.973+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 msgr2=0x55a2bed92640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.973+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bed92640 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f400983e0 tx=0x7f8f4009c0c0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 shutdown_connections 2024-09-10T10:04:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2beda5180 0x55a2beda7600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bed92640 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed92100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2732271591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed91bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 >> 172.21.15.45:0/2732271591 conn(0x55a2bed86060 msgr2=0x55a2bed8bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f26ffd640 0 client.24898 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.974+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 shutdown_connections 2024-09-10T10:04:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.975+0000 7f8f5080b740 1 -- 172.21.15.45:0/2732271591 wait complete. 2024-09-10T10:04:48.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:48.975+0000 7f8f5080b740 20 client.24898 trim_cache size 0 max 16384 2024-09-10T10:04:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.050+0000 7f8f5080b740 1 Processor -- start 2024-09-10T10:04:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.050+0000 7f8f5080b740 1 -- start start 2024-09-10T10:04:49.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.051+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed89af0 0x55a2bed93b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.051+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed8a410 0x55a2bed92470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.051+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bedacf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.052+0000 7f8f5080b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400c690 con 0x55a2bed8a410 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.052+0000 7f8f5080b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400ee60 con 0x55a2bed89af0 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.052+0000 7f8f5080b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400cc50 con 0x55a2bed94cb0 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.052+0000 7f8f46ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed89af0 0x55a2bed93b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.052+0000 7f8f46ffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed89af0 0x55a2bed93b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:53966/0 (socket says 172.21.15.45:53966) 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.052+0000 7f8f46ffd640 1 -- 172.21.15.45:0/1234964347 learned_addr learned my addr 172.21.15.45:0/1234964347 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.052+0000 7f8f477fe640 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bedacf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.054+0000 7f8f467fc640 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed8a410 0x55a2bed92470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.056+0000 7f8f467fc640 1 -- 172.21.15.45:0/1234964347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 msgr2=0x55a2bedacf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.056+0000 7f8f467fc640 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bedacf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.056+0000 7f8f467fc640 1 -- 172.21.15.45:0/1234964347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed89af0 msgr2=0x55a2bed93b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.056+0000 7f8f467fc640 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed89af0 0x55a2bed93b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.056+0000 7f8f467fc640 1 -- 172.21.15.45:0/1234964347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2bed90b50 con 0x55a2bed8a410 2024-09-10T10:04:49.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.057+0000 7f8f467fc640 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed8a410 0x55a2bed92470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00b4b0 tx=0x7f8f3c009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f84e11f3ba82e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.057+0000 7f8f2d7fa640 1 -- 172.21.15.45:0/1234964347 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c015030 con 0x55a2bed8a410 2024-09-10T10:04:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.058+0000 7f8f2d7fa640 1 -- 172.21.15.45:0/1234964347 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f3c01e070 con 0x55a2bed8a410 2024-09-10T10:04:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.058+0000 7f8f2d7fa640 1 -- 172.21.15.45:0/1234964347 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0038f0 con 0x55a2bed8a410 2024-09-10T10:04:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.060+0000 7f8f5080b740 1 -- 172.21.15.45:0/1234964347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed8a410 msgr2=0x55a2bed92470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.060+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed8a410 0x55a2bed92470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00b4b0 tx=0x7f8f3c009320 comp rx=0 tx=0).stop 2024-09-10T10:04:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.061+0000 7f8f5080b740 1 -- 172.21.15.45:0/1234964347 shutdown_connections 2024-09-10T10:04:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.061+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bedacf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.061+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed8a410 0x55a2bed92470 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.061+0000 7f8f5080b740 1 --2- 172.21.15.45:0/1234964347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed89af0 0x55a2bed93b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.061+0000 7f8f5080b740 1 -- 172.21.15.45:0/1234964347 >> 172.21.15.45:0/1234964347 conn(0x55a2bed86060 msgr2=0x55a2bed93420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.063+0000 7f8f5080b740 1 -- 172.21.15.45:0/1234964347 shutdown_connections 2024-09-10T10:04:49.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.063+0000 7f8f5080b740 1 -- 172.21.15.45:0/1234964347 wait complete. 2024-09-10T10:04:49.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.064+0000 7f8f5080b740 1 -- start start 2024-09-10T10:04:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.065+0000 7f8f5080b740 1 Processor -- start 2024-09-10T10:04:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.066+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.067+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.067+0000 7f8f5080b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bee0ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.067+0000 7f8f5080b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400d320 con 0x55a2bed89af0 2024-09-10T10:04:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.067+0000 7f8f5080b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400e790 con 0x55a2bed8a410 2024-09-10T10:04:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.067+0000 7f8f5080b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3400d9f0 con 0x55a2bed94cb0 2024-09-10T10:04:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.068+0000 7f8f5080b740 10 client.0 _ll_register_callbacks cb 0x7f8f40098220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.070+0000 7f8f46ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.070+0000 7f8f46ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed98270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44068/0 (socket says 172.21.15.45:44068) 2024-09-10T10:04:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.070+0000 7f8f46ffd640 1 -- 172.21.15.45:0/2136686547 learned_addr learned my addr 172.21.15.45:0/2136686547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:04:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.070+0000 7f8f467fc640 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.070+0000 7f8f477fe640 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bee0ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.071+0000 7f8f46ffd640 1 -- 172.21.15.45:0/2136686547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 msgr2=0x55a2bee0ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.071+0000 7f8f46ffd640 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bee0ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.071+0000 7f8f46ffd640 1 -- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 msgr2=0x55a2bed987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.071+0000 7f8f46ffd640 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.071+0000 7f8f46ffd640 1 -- 172.21.15.45:0/2136686547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2bee0e510 con 0x55a2bed89af0 2024-09-10T10:04:49.076 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.073+0000 7f8f46ffd640 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed98270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005dc0 tx=0x7f8f380035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb1946b744b5d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.083+0000 7f8f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:04:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.083+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2136686547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38056060 con 0x55a2bed89af0 2024-09-10T10:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.083+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f3c01e070 con 0x55a2bed89af0 2024-09-10T10:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.083+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2136686547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f8f38031740 con 0x55a2bed89af0 2024-09-10T10:04:49.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.094+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2136686547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8f38035040 con 0x55a2bed89af0 2024-09-10T10:04:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.094+0000 7f8f5080b740 10 client.15270 fetch_fsmap learned FSMap version 19 2024-09-10T10:04:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.094+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a2bee0f800 con 0x55a2bed89af0 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.108+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2136686547 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f38040070 con 0x55a2bed89af0 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.108+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2136686547 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f8f38048c30 con 0x55a2bed89af0 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.109+0000 7f8f5080b740 10 client.15270 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.109+0000 7f8f5080b740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.109+0000 7f8f5080b740 20 client.15270 populate_metadata read hostname 'smithi045' 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.109+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2bee0ff60 0x55a2bee12380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.109+0000 7f8f5080b740 4 client.15270 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:04:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.109+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8f3c01e070 con 0x55a2bee0ff60 2024-09-10T10:04:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.110+0000 7f8f467fc640 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2bee0ff60 0x55a2bee12380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.112+0000 7f8f467fc640 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2bee0ff60 0x55a2bee12380 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d37dfc9e9ae717 server_cookie=fa653d7fafc019fb in_seq=0 out_seq=0 2024-09-10T10:04:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:49.112+0000 7f8f26ffd640 10 client.15270 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:49.401 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:48 smithi162 ceph-mon[34963]: pgmap v51: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1001 MiB/s wr, 2.67k op/s 2024-09-10T10:04:49.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:48 smithi045 ceph-mon[31114]: pgmap v51: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1001 MiB/s wr, 2.67k op/s 2024-09-10T10:04:49.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:48 smithi047 ceph-mon[35244]: pgmap v51: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1001 MiB/s wr, 2.67k op/s 2024-09-10T10:04:51.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:50 smithi162 ceph-mon[34963]: pgmap v52: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 799 MiB/s wr, 1.87k op/s 2024-09-10T10:04:51.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:50 smithi047 ceph-mon[35244]: pgmap v52: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 799 MiB/s wr, 1.87k op/s 2024-09-10T10:04:51.416 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:50 smithi045 ceph-mon[31114]: pgmap v52: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 799 MiB/s wr, 1.87k op/s 2024-09-10T10:04:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.638+0000 7f8f26ffd640 1 -- 172.21.15.45:0/2136686547 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8f3c01e070 con 0x55a2bee0ff60 2024-09-10T10:04:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.638+0000 7f8f26ffd640 10 client.15270 handle_command_reply: tid=0 2024-09-10T10:04:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.638+0000 7f8f5080b740 1 client.15270 shutdown 2024-09-10T10:04:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 msgr2=0x55a2bed98270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed98270 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005dc0 tx=0x7f8f380035f0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 shutdown_connections 2024-09-10T10:04:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a2bee0ff60 0x55a2bee12380 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a2bed94cb0 0x55a2bee0ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2bed8a410 0x55a2bed987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 --2- 172.21.15.45:0/2136686547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a2bed89af0 0x55a2bed98270 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 >> 172.21.15.45:0/2136686547 conn(0x55a2bed86060 msgr2=0x55a2bed851c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f26ffd640 0 client.15270 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:04:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 shutdown_connections 2024-09-10T10:04:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.639+0000 7f8f5080b740 1 -- 172.21.15.45:0/2136686547 wait complete. 2024-09-10T10:04:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.640+0000 7f8f5080b740 20 client.15270 trim_cache size 0 max 16384 2024-09-10T10:04:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:04:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:04:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:04:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:04:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.641+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8f300a2fd0 msgr2=0x7f8f300a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.642+0000 7f8f4d991640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8f300a2fd0 0x7f8f300a5490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f38005c20 tx=0x7f8f38000bc0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.642+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 msgr2=0x7f8f48114ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.642+0000 7f8f4d991640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 0x7f8f48114ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b990 tx=0x7f8f4000be50 comp rx=0 tx=0).stop 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.643+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 shutdown_connections 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.643+0000 7f8f4d991640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8f4811b140 0x7f8f48114ea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.643+0000 7f8f4d991640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8f300a2fd0 0x7f8f300a5490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.643+0000 7f8f4d991640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f481095d0 0x7f8f48114960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.643+0000 7f8f4d991640 1 --2- 172.21.15.45:0/1287197741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f481091f0 0x7f8f4811a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.643+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 >> 172.21.15.45:0/1287197741 conn(0x7f8f4806ce00 msgr2=0x7f8f480709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.644+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 shutdown_connections 2024-09-10T10:04:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:04:52.644+0000 7f8f4d991640 1 -- 172.21.15.45:0/1287197741 wait complete. 2024-09-10T10:04:53.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:53 smithi162 ceph-mon[34963]: pgmap v53: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 727 MiB/s wr, 1.66k op/s 2024-09-10T10:04:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:53 smithi045 ceph-mon[31114]: pgmap v53: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 727 MiB/s wr, 1.66k op/s 2024-09-10T10:04:53.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:53 smithi047 ceph-mon[35244]: pgmap v53: 97 pgs: 97 active+clean; 10 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 727 MiB/s wr, 1.66k op/s 2024-09-10T10:04:55.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:55 smithi162 ceph-mon[34963]: pgmap v54: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 GiB/s wr, 2.59k op/s 2024-09-10T10:04:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:55 smithi045 ceph-mon[31114]: pgmap v54: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 GiB/s wr, 2.59k op/s 2024-09-10T10:04:55.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:55 smithi047 ceph-mon[35244]: pgmap v54: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.0 GiB/s wr, 2.59k op/s 2024-09-10T10:04:57.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:57 smithi162 ceph-mon[34963]: pgmap v55: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.66k op/s 2024-09-10T10:04:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:57 smithi045 ceph-mon[31114]: pgmap v55: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.66k op/s 2024-09-10T10:04:57.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:57 smithi047 ceph-mon[35244]: pgmap v55: 97 pgs: 97 active+clean; 10 GiB data, 48 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 694 MiB/s wr, 1.66k op/s 2024-09-10T10:04:59.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:04:59 smithi162 ceph-mon[34963]: pgmap v56: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 888 MiB/s wr, 2.10k op/s 2024-09-10T10:04:59.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:04:59 smithi045 ceph-mon[31114]: pgmap v56: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 888 MiB/s wr, 2.10k op/s 2024-09-10T10:04:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:04:59 smithi047 ceph-mon[35244]: pgmap v56: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 888 MiB/s wr, 2.10k op/s 2024-09-10T10:05:01.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:01 smithi162 ceph-mon[34963]: pgmap v57: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 761 MiB/s wr, 1.88k op/s 2024-09-10T10:05:01.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:01 smithi045 ceph-mon[31114]: pgmap v57: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 761 MiB/s wr, 1.88k op/s 2024-09-10T10:05:01.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:01 smithi047 ceph-mon[35244]: pgmap v57: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 761 MiB/s wr, 1.88k op/s 2024-09-10T10:05:03.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:03 smithi162 ceph-mon[34963]: pgmap v58: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 688 MiB/s wr, 1.73k op/s 2024-09-10T10:05:03.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:03 smithi045 ceph-mon[31114]: pgmap v58: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 688 MiB/s wr, 1.73k op/s 2024-09-10T10:05:03.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:03 smithi047 ceph-mon[35244]: pgmap v58: 97 pgs: 97 active+clean; 10 GiB data, 52 GiB used, 1021 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 688 MiB/s wr, 1.73k op/s 2024-09-10T10:05:05.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:05 smithi162 ceph-mon[34963]: pgmap v59: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 3.42k op/s 2024-09-10T10:05:05.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:05 smithi045 ceph-mon[31114]: pgmap v59: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 3.42k op/s 2024-09-10T10:05:05.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:05 smithi047 ceph-mon[35244]: pgmap v59: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 3.42k op/s 2024-09-10T10:05:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.942+0000 7f417ff4b640 1 -- 172.21.15.45:0/1033472493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 msgr2=0x7f4178110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.942+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1033472493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 0x7f4178110250 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4170009ca0 tx=0x7f417002a8e0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.945+0000 7f417ff4b640 1 -- 172.21.15.45:0/1033472493 shutdown_connections 2024-09-10T10:05:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.945+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1033472493 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f417811b140 0x7f417811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.945+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1033472493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41781095d0 0x7f4178109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.945+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1033472493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 0x7f4178110250 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.945+0000 7f417ff4b640 1 -- 172.21.15.45:0/1033472493 >> 172.21.15.45:0/1033472493 conn(0x7f417806ce00 msgr2=0x7f417806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:05.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.946+0000 7f417ff4b640 1 -- 172.21.15.45:0/1033472493 shutdown_connections 2024-09-10T10:05:05.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.946+0000 7f417ff4b640 1 -- 172.21.15.45:0/1033472493 wait complete. 2024-09-10T10:05:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.947+0000 7f417ff4b640 1 Processor -- start 2024-09-10T10:05:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.947+0000 7f417ff4b640 1 -- start start 2024-09-10T10:05:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.956+0000 7f417ff4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 0x7f4178118610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.956+0000 7f417ff4b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41781095d0 0x7f4178118b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.956+0000 7f417ff4b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417811b140 0x7f41781117d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.956+0000 7f417ff4b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4178120d30 con 0x7f41781091f0 2024-09-10T10:05:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.956+0000 7f417ff4b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4178120bb0 con 0x7f417811b140 2024-09-10T10:05:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.956+0000 7f417ff4b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f4178120eb0 con 0x7f41781095d0 2024-09-10T10:05:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.956+0000 7f417dce7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 0x7f4178118610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.957+0000 7f417dce7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 0x7f4178118610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42538/0 (socket says 172.21.15.45:42538) 2024-09-10T10:05:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.957+0000 7f417dce7640 1 -- 172.21.15.45:0/1067118986 learned_addr learned my addr 172.21.15.45:0/1067118986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.957+0000 7f417e4e8640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417811b140 0x7f41781117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.958+0000 7f417e4e8640 1 -- 172.21.15.45:0/1067118986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41781095d0 msgr2=0x7f4178118b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.958+0000 7f417e4e8640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41781095d0 0x7f4178118b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.958+0000 7f417e4e8640 1 -- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 msgr2=0x7f4178118610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.958+0000 7f417e4e8640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 0x7f4178118610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.958+0000 7f417e4e8640 1 -- 172.21.15.45:0/1067118986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4178112090 con 0x7f417811b140 2024-09-10T10:05:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.959+0000 7f417e4e8640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417811b140 0x7f41781117d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f416800c710 tx=0x7f416800cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.964+0000 7f416effd640 1 -- 172.21.15.45:0/1067118986 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168004220 con 0x7f417811b140 2024-09-10T10:05:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.965+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4178112380 con 0x7f417811b140 2024-09-10T10:05:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.965+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4178116360 con 0x7f417811b140 2024-09-10T10:05:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.970+0000 7f4182dc3740 1 Processor -- start 2024-09-10T10:05:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.970+0000 7f416effd640 1 -- 172.21.15.45:0/1067118986 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f416800ce20 con 0x7f417811b140 2024-09-10T10:05:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.970+0000 7f416effd640 1 -- 172.21.15.45:0/1067118986 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168003e20 con 0x7f417811b140 2024-09-10T10:05:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.975+0000 7f4182dc3740 1 -- start start 2024-09-10T10:05:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.975+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105420ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.975+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105421410 0x55a10542b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.975+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a10542bcb0 0x55a10542e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.975+0000 7f4182dc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a1053852f0 con 0x55a105421410 2024-09-10T10:05:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.975+0000 7f4182dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a10532d270 con 0x55a10542bcb0 2024-09-10T10:05:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.975+0000 7f4182dc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a1053812b0 con 0x55a105420af0 2024-09-10T10:05:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.976+0000 7f416effd640 1 -- 172.21.15.45:0/1067118986 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f416800f550 con 0x7f417811b140 2024-09-10T10:05:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.977+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a10542bcb0 0x55a10542e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.978+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105420ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.978+0000 7f416effd640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41640a2fd0 0x7f41640a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.978+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105420ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:49486/0 (socket says 172.21.15.45:49486) 2024-09-10T10:05:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.978+0000 7f417d4e6640 1 -- 172.21.15.45:0/3145157741 learned_addr learned my addr 172.21.15.45:0/3145157741 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.979+0000 7f416effd640 1 -- 172.21.15.45:0/1067118986 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f41680cee90 con 0x7f417811b140 2024-09-10T10:05:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.985+0000 7f417e4e8640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41640a2fd0 0x7f41640a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.986+0000 7f417d4e6640 1 -- 172.21.15.45:0/3145157741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a10542bcb0 msgr2=0x55a10542e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.986+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3145157741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a10542bcb0 0x55a10542e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.986+0000 7f417d4e6640 1 -- 172.21.15.45:0/3145157741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105421410 msgr2=0x55a10542b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.986+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3145157741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105421410 0x55a10542b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.986+0000 7f417d4e6640 1 -- 172.21.15.45:0/3145157741 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10532c340 con 0x55a105420af0 2024-09-10T10:05:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.993+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3145157741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105420ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4174015420 tx=0x7f4174000fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8773c8b0e72ceca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.995+0000 7f417e4e8640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41640a2fd0 0x7f41640a5490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4168097d50 tx=0x7f41680cf0e0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.996+0000 7f4163fff640 1 -- 172.21.15.45:0/3145157741 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174010040 con 0x55a105420af0 2024-09-10T10:05:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.996+0000 7f4163fff640 1 -- 172.21.15.45:0/3145157741 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417400b070 con 0x55a105420af0 2024-09-10T10:05:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.998+0000 7f4182dc3740 1 -- 172.21.15.45:0/3145157741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 msgr2=0x55a105420ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.998+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3145157741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105420ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4174015420 tx=0x7f4174000fd0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.998+0000 7f4182dc3740 1 -- 172.21.15.45:0/3145157741 shutdown_connections 2024-09-10T10:05:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.998+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3145157741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a10542bcb0 0x55a10542e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.998+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3145157741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105421410 0x55a10542b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.998+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3145157741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105420ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.998+0000 7f4182dc3740 1 -- 172.21.15.45:0/3145157741 >> 172.21.15.45:0/3145157741 conn(0x55a10541d060 msgr2=0x55a10541d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:05.999+0000 7f4182dc3740 1 -- 172.21.15.45:0/3145157741 shutdown_connections 2024-09-10T10:05:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.000+0000 7f4182dc3740 1 -- 172.21.15.45:0/3145157741 wait complete. 2024-09-10T10:05:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.001+0000 7f4182dc3740 1 -- start start 2024-09-10T10:05:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.001+0000 7f4182dc3740 1 Processor -- start 2024-09-10T10:05:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.002+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105428ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.002+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a105428ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.002+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a10542bcb0 0x55a105429530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.002+0000 7f4182dc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a105450930 con 0x55a10542bcb0 2024-09-10T10:05:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.002+0000 7f4182dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a10530e290 con 0x55a105421410 2024-09-10T10:05:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.002+0000 7f4182dc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a105450ab0 con 0x55a105420af0 2024-09-10T10:05:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.002+0000 7f4182dc3740 10 client.0 _ll_register_callbacks cb 0x55a10533f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.003+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a10542bcb0 0x55a105429530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.003+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a10542bcb0 0x55a105429530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42554/0 (socket says 172.21.15.45:42554) 2024-09-10T10:05:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.003+0000 7f417d4e6640 1 -- 172.21.15.45:0/3717932488 learned_addr learned my addr 172.21.15.45:0/3717932488 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.003+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105428ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.005+0000 7f417d4e6640 1 -- 172.21.15.45:0/3717932488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 msgr2=0x55a105428ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.005+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105428ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.005+0000 7f417d4e6640 1 -- 172.21.15.45:0/3717932488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 msgr2=0x55a105428ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.005+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a105428ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.005+0000 7f417d4e6640 1 -- 172.21.15.45:0/3717932488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a105098210 con 0x55a10542bcb0 2024-09-10T10:05:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.006+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a10542bcb0 0x55a105429530 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f41740017f0 tx=0x7f4174007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bdf9361170bed57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.008+0000 7f41617fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:05:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.008+0000 7f41617fa640 1 -- 172.21.15.45:0/3717932488 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174004220 con 0x55a10542bcb0 2024-09-10T10:05:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.008+0000 7f41617fa640 1 -- 172.21.15.45:0/3717932488 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f41740043c0 con 0x55a10542bcb0 2024-09-10T10:05:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.009+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f417400b070 con 0x55a10542bcb0 2024-09-10T10:05:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.011+0000 7f41617fa640 1 -- 172.21.15.45:0/3717932488 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4174001d00 con 0x55a10542bcb0 2024-09-10T10:05:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.013+0000 7f4182dc3740 10 client.15279 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.013+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a105164620 con 0x55a10542bcb0 2024-09-10T10:05:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.015+0000 7f41617fa640 1 -- 172.21.15.45:0/3717932488 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417400b070 con 0x55a10542bcb0 2024-09-10T10:05:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.016+0000 7f41617fa640 1 -- 172.21.15.45:0/3717932488 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f4174032470 con 0x55a10542bcb0 2024-09-10T10:05:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.017+0000 7f4182dc3740 10 client.15279 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.017+0000 7f4182dc3740 10 client.15279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.017+0000 7f4182dc3740 20 client.15279 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:06.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.017+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a10545dbf0 0x55a10545ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.017+0000 7f4182dc3740 4 client.15279 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:05:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.017+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a105392a20 con 0x55a10545dbf0 2024-09-10T10:05:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.019+0000 7f417dce7640 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a10545dbf0 0x55a10545ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.027+0000 7f417dce7640 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a10545dbf0 0x55a10545ffe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69c975fcabc5832 server_cookie=4cc8952ad65174ca in_seq=0 out_seq=0 2024-09-10T10:05:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.027+0000 7f41617fa640 10 client.15279 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.032+0000 7f41617fa640 1 -- 172.21.15.45:0/3717932488 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a105392a20 con 0x55a10545dbf0 2024-09-10T10:05:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.032+0000 7f41617fa640 10 client.15279 handle_command_reply: tid=0 2024-09-10T10:05:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.033+0000 7f4182dc3740 1 client.15279 shutdown 2024-09-10T10:05:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.033+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a10542bcb0 msgr2=0x55a105429530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.033+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a10542bcb0 0x55a105429530 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f41740017f0 tx=0x7f4174007bb0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 shutdown_connections 2024-09-10T10:05:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a10545dbf0 0x55a10545ffe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a10542bcb0 0x55a105429530 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a105428ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3717932488 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105428ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 >> 172.21.15.45:0/3717932488 conn(0x55a10541d060 msgr2=0x55a105422dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f41617fa640 0 client.15279 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.035+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 shutdown_connections 2024-09-10T10:05:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.036+0000 7f4182dc3740 1 -- 172.21.15.45:0/3717932488 wait complete. 2024-09-10T10:05:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.036+0000 7f4182dc3740 20 client.15279 trim_cache size 0 max 16384 2024-09-10T10:05:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.106+0000 7f4182dc3740 1 Processor -- start 2024-09-10T10:05:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.109+0000 7f4182dc3740 1 -- start start 2024-09-10T10:05:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.109+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105420af0 0x55a105428370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.109+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a10542a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.109+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105465700 0x55a105465b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.109+0000 7f4182dc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f415c00c7d0 con 0x55a105420af0 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.109+0000 7f4182dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f415c00f070 con 0x55a105421410 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.109+0000 7f4182dc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f415c00ce60 con 0x55a105465700 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.110+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105420af0 0x55a105428370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.111+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105420af0 0x55a105428370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42558/0 (socket says 172.21.15.45:42558) 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.111+0000 7f417d4e6640 1 -- 172.21.15.45:0/3017077854 learned_addr learned my addr 172.21.15.45:0/3017077854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.111+0000 7f417d4e6640 1 -- 172.21.15.45:0/3017077854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105465700 msgr2=0x55a105465b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.111+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3017077854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105465700 0x55a105465b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.111+0000 7f417d4e6640 1 -- 172.21.15.45:0/3017077854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 msgr2=0x55a10542a310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.111+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3017077854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a10542a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.111+0000 7f417d4e6640 1 -- 172.21.15.45:0/3017077854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10541ce90 con 0x55a105420af0 2024-09-10T10:05:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.112+0000 7f417d4e6640 1 --2- 172.21.15.45:0/3017077854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105420af0 0x55a105428370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4174019980 tx=0x7f4174001c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e013da3eee6ec98c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.118+0000 7f41637fe640 1 -- 172.21.15.45:0/3017077854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174010030 con 0x55a105420af0 2024-09-10T10:05:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.118+0000 7f41637fe640 1 -- 172.21.15.45:0/3017077854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4174037070 con 0x55a105420af0 2024-09-10T10:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.119+0000 7f4182dc3740 1 -- 172.21.15.45:0/3017077854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105420af0 msgr2=0x55a105428370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.119+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3017077854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105420af0 0x55a105428370 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4174019980 tx=0x7f4174001c80 comp rx=0 tx=0).stop 2024-09-10T10:05:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.123+0000 7f4182dc3740 1 -- 172.21.15.45:0/3017077854 shutdown_connections 2024-09-10T10:05:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.123+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3017077854 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105465700 0x55a105465b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.123+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3017077854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a10542a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.123+0000 7f4182dc3740 1 --2- 172.21.15.45:0/3017077854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105420af0 0x55a105428370 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.123+0000 7f4182dc3740 1 -- 172.21.15.45:0/3017077854 >> 172.21.15.45:0/3017077854 conn(0x55a10541d060 msgr2=0x55a105422dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.127+0000 7f4182dc3740 1 -- 172.21.15.45:0/3017077854 shutdown_connections 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.127+0000 7f4182dc3740 1 -- 172.21.15.45:0/3017077854 wait complete. 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 -- start start 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 Processor -- start 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105455e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a105456370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105465700 0x55a105450de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f415c00d530 con 0x55a105465700 2024-09-10T10:05:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f415c00e9a0 con 0x55a105421410 2024-09-10T10:05:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f415c00dc00 con 0x55a105420af0 2024-09-10T10:05:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.128+0000 7f4182dc3740 10 client.0 _ll_register_callbacks cb 0x7f41740019a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.129+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105455e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.129+0000 7f417d4e6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105455e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:49510/0 (socket says 172.21.15.45:49510) 2024-09-10T10:05:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.129+0000 7f417d4e6640 1 -- 172.21.15.45:0/4062836247 learned_addr learned my addr 172.21.15.45:0/4062836247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.130+0000 7f417d4e6640 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105465700 0x55a105450de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.131+0000 7f417d4e6640 1 -- 172.21.15.45:0/4062836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 msgr2=0x55a105455e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.131+0000 7f417d4e6640 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105455e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.131+0000 7f417d4e6640 1 -- 172.21.15.45:0/4062836247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 msgr2=0x55a105456370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.131+0000 7f417d4e6640 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a105456370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.131+0000 7f417d4e6640 1 -- 172.21.15.45:0/4062836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a105451320 con 0x55a105465700 2024-09-10T10:05:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.132+0000 7f417d4e6640 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105465700 0x55a105450de0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4174015d70 tx=0x7f4174000e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1b84d57eee05b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.136+0000 7f41617fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:05:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.136+0000 7f41617fa640 1 -- 172.21.15.45:0/4062836247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4174010070 con 0x55a105465700 2024-09-10T10:05:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.136+0000 7f41617fa640 1 -- 172.21.15.45:0/4062836247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f417403c460 con 0x55a105465700 2024-09-10T10:05:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.136+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4174037070 con 0x55a105465700 2024-09-10T10:05:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.137+0000 7f41617fa640 1 -- 172.21.15.45:0/4062836247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f4174032cc0 con 0x55a105465700 2024-09-10T10:05:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.138+0000 7f4182dc3740 10 client.15285 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.138+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a1054c8d70 con 0x55a105465700 2024-09-10T10:05:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f41617fa640 1 -- 172.21.15.45:0/4062836247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4174037070 con 0x55a105465700 2024-09-10T10:05:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f41617fa640 1 -- 172.21.15.45:0/4062836247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f41740285a0 con 0x55a105465700 2024-09-10T10:05:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f4182dc3740 10 client.15285 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f4182dc3740 10 client.15285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f4182dc3740 20 client.15285 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f4182dc3740 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a1054c93a0 0x55a1054cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f4182dc3740 4 client.15285 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:05:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.144+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a10538cd70 con 0x55a1054c93a0 2024-09-10T10:05:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.146+0000 7f417dce7640 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a1054c93a0 0x55a1054cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.148+0000 7f417dce7640 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a1054c93a0 0x55a1054cb7c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f67833b8c13a6d6e server_cookie=17704d7b3e1cf1e5 in_seq=0 out_seq=0 2024-09-10T10:05:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:06.149+0000 7f41617fa640 10 client.15285 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:07.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:07 smithi162 ceph-mon[34963]: pgmap v60: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 694 MiB/s wr, 2.48k op/s 2024-09-10T10:05:07.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:07 smithi045 ceph-mon[31114]: pgmap v60: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 694 MiB/s wr, 2.48k op/s 2024-09-10T10:05:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:07 smithi047 ceph-mon[35244]: pgmap v60: 97 pgs: 97 active+clean; 10 GiB data, 55 GiB used, 1017 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 694 MiB/s wr, 2.48k op/s 2024-09-10T10:05:09.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.005+0000 7f41617fa640 1 -- 172.21.15.45:0/4062836247 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a10538cd70 con 0x55a1054c93a0 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.005+0000 7f41617fa640 10 client.15285 handle_command_reply: tid=0 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.005+0000 7f4182dc3740 1 client.15285 shutdown 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105465700 msgr2=0x55a105450de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105465700 0x55a105450de0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4174015d70 tx=0x7f4174000e80 comp rx=0 tx=0).stop 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 shutdown_connections 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a1054c93a0 0x55a1054cb7c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a105465700 0x55a105450de0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a105421410 0x55a105456370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 --2- 172.21.15.45:0/4062836247 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a105420af0 0x55a105455e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.008 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 >> 172.21.15.45:0/4062836247 conn(0x55a10541d060 msgr2=0x55a105464f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f41617fa640 0 client.15285 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.006+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 shutdown_connections 2024-09-10T10:05:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.007+0000 7f4182dc3740 1 -- 172.21.15.45:0/4062836247 wait complete. 2024-09-10T10:05:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.007+0000 7f4182dc3740 20 client.15285 trim_cache size 0 max 16384 2024-09-10T10:05:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:05:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:05:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:05:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:05:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.012+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41640a2fd0 msgr2=0x7f41640a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.012+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41640a2fd0 0x7f41640a5490 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4168097d50 tx=0x7f41680cf0e0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.012+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417811b140 msgr2=0x7f41781117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.012+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417811b140 0x7f41781117d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f416800c710 tx=0x7f416800cbd0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.013+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 shutdown_connections 2024-09-10T10:05:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.013+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41640a2fd0 0x7f41640a5490 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.013+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f417811b140 0x7f41781117d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.013+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41781095d0 0x7f4178118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.013+0000 7f417ff4b640 1 --2- 172.21.15.45:0/1067118986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41781091f0 0x7f4178118610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.014+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 >> 172.21.15.45:0/1067118986 conn(0x7f417806ce00 msgr2=0x7f417811b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.014+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 shutdown_connections 2024-09-10T10:05:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:09.014+0000 7f417ff4b640 1 -- 172.21.15.45:0/1067118986 wait complete. 2024-09-10T10:05:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:09 smithi045 ceph-mon[31114]: pgmap v61: 97 pgs: 97 active+clean; 11 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 815 MiB/s wr, 3.04k op/s 2024-09-10T10:05:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:09 smithi047 ceph-mon[35244]: pgmap v61: 97 pgs: 97 active+clean; 11 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 815 MiB/s wr, 3.04k op/s 2024-09-10T10:05:09.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:09 smithi162 ceph-mon[34963]: pgmap v61: 97 pgs: 97 active+clean; 11 GiB data, 58 GiB used, 1014 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 815 MiB/s wr, 3.04k op/s 2024-09-10T10:05:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:11 smithi047 ceph-mon[35244]: pgmap v62: 97 pgs: 97 active+clean; 12 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 MiB/s wr, 3.58k op/s 2024-09-10T10:05:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:11 smithi162 ceph-mon[34963]: pgmap v62: 97 pgs: 97 active+clean; 12 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 MiB/s wr, 3.58k op/s 2024-09-10T10:05:11.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:11 smithi045 ceph-mon[31114]: pgmap v62: 97 pgs: 97 active+clean; 12 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 841 MiB/s wr, 3.58k op/s 2024-09-10T10:05:13.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:13 smithi162 ceph-mon[34963]: pgmap v63: 97 pgs: 97 active+clean; 12 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 688 MiB/s wr, 3.21k op/s 2024-09-10T10:05:13.905 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:13 smithi045 ceph-mon[31114]: pgmap v63: 97 pgs: 97 active+clean; 12 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 688 MiB/s wr, 3.21k op/s 2024-09-10T10:05:13.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:13 smithi047 ceph-mon[35244]: pgmap v63: 97 pgs: 97 active+clean; 12 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 688 MiB/s wr, 3.21k op/s 2024-09-10T10:05:15.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:15 smithi162 ceph-mon[34963]: pgmap v64: 97 pgs: 97 active+clean; 15 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 GiB/s wr, 4.94k op/s 2024-09-10T10:05:15.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:15 smithi045 ceph-mon[31114]: pgmap v64: 97 pgs: 97 active+clean; 15 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 GiB/s wr, 4.94k op/s 2024-09-10T10:05:15.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:15 smithi047 ceph-mon[35244]: pgmap v64: 97 pgs: 97 active+clean; 15 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.0 GiB/s wr, 4.94k op/s 2024-09-10T10:05:16.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:16 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:05:16.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:16 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:05:16.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:16 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:05:17.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:17 smithi162 ceph-mon[34963]: pgmap v65: 97 pgs: 97 active+clean; 15 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 687 MiB/s wr, 3.26k op/s 2024-09-10T10:05:17.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:17 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:17 smithi045 ceph-mon[31114]: pgmap v65: 97 pgs: 97 active+clean; 15 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 687 MiB/s wr, 3.26k op/s 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:17 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:17 smithi047 ceph-mon[35244]: pgmap v65: 97 pgs: 97 active+clean; 15 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 687 MiB/s wr, 3.26k op/s 2024-09-10T10:05:17.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:17.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:17 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:19.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:18 smithi162 ceph-mon[34963]: pgmap v66: 97 pgs: 97 active+clean; 16 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 795 MiB/s wr, 3.40k op/s 2024-09-10T10:05:19.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:18 smithi045 ceph-mon[31114]: pgmap v66: 97 pgs: 97 active+clean; 16 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 795 MiB/s wr, 3.40k op/s 2024-09-10T10:05:19.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:18 smithi047 ceph-mon[35244]: pgmap v66: 97 pgs: 97 active+clean; 16 GiB data, 75 GiB used, 998 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 795 MiB/s wr, 3.40k op/s 2024-09-10T10:05:20.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:19 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:05:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:19 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:05:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:19 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:20.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:19 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:05:20.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:19 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:05:20.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:19 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:20.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:19 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:05:20.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:19 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:05:20.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:19 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:05:21.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:20 smithi162 ceph-mon[34963]: pgmap v67: 97 pgs: 97 active+clean; 18 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 906 MiB/s wr, 3.13k op/s 2024-09-10T10:05:21.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:20 smithi045 ceph-mon[31114]: pgmap v67: 97 pgs: 97 active+clean; 18 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 906 MiB/s wr, 3.13k op/s 2024-09-10T10:05:21.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:20 smithi047 ceph-mon[35244]: pgmap v67: 97 pgs: 97 active+clean; 18 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 906 MiB/s wr, 3.13k op/s 2024-09-10T10:05:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.321+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/622433078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c0952d0 msgr2=0x7fb59c0a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.321+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/622433078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c0952d0 0x7fb59c0a1370 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4067960 tx=0x7fb5a4096c30 comp rx=0 tx=0).stop 2024-09-10T10:05:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.322+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/622433078 shutdown_connections 2024-09-10T10:05:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.322+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/622433078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c0952d0 0x7fb59c0a1370 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.322+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/622433078 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59c094930 0x7fb59c094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.322+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/622433078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c09a5b0 0x7fb59c09a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.322+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/622433078 >> 172.21.15.45:0/622433078 conn(0x7fb59c00a220 msgr2=0x7fb59c00a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.322+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/622433078 shutdown_connections 2024-09-10T10:05:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.322+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/622433078 wait complete. 2024-09-10T10:05:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 Processor -- start 2024-09-10T10:05:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 -- start start 2024-09-10T10:05:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c094930 0x7fb59c12be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c0952d0 0x7fb59c12c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59c09a5b0 0x7fb59c130730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0a4850 con 0x7fb59c094930 2024-09-10T10:05:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0a46d0 con 0x7fb59c0952d0 2024-09-10T10:05:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5aa6ca640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c0a49d0 con 0x7fb59c09a5b0 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5a96c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c094930 0x7fb59c12be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5a96c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c094930 0x7fb59c12be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48006/0 (socket says 172.21.15.45:48006) 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.323+0000 7fb5a96c8640 1 -- 172.21.15.45:0/1865222132 learned_addr learned my addr 172.21.15.45:0/1865222132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.324+0000 7fb5a96c8640 1 -- 172.21.15.45:0/1865222132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59c09a5b0 msgr2=0x7fb59c130730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.324+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59c09a5b0 0x7fb59c130730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.324+0000 7fb5a96c8640 1 -- 172.21.15.45:0/1865222132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c0952d0 msgr2=0x7fb59c12c3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.324+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c0952d0 0x7fb59c12c3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.324+0000 7fb5a96c8640 1 -- 172.21.15.45:0/1865222132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59c130e30 con 0x7fb59c094930 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.324+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c094930 0x7fb59c12be70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000b4a0 tx=0x7fb5a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.325+0000 7fb59a7fc640 1 -- 172.21.15.45:0/1865222132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0014070 con 0x7fb59c094930 2024-09-10T10:05:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.325+0000 7fb59a7fc640 1 -- 172.21.15.45:0/1865222132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a0010af0 con 0x7fb59c094930 2024-09-10T10:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.326+0000 7fb59a7fc640 1 -- 172.21.15.45:0/1865222132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0017d10 con 0x7fb59c094930 2024-09-10T10:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.327+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59c131120 con 0x7fb59c094930 2024-09-10T10:05:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.327+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59c096090 con 0x7fb59c094930 2024-09-10T10:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 Processor -- start 2024-09-10T10:05:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 -- start start 2024-09-10T10:05:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a5609d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 0x559a560a7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb59a7fc640 1 -- 172.21.15.45:0/1865222132 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb5a0017070 con 0x7fb59c094930 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560aa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48014/0 (socket says 172.21.15.45:48014) 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/4029062267 learned_addr learned my addr 172.21.15.45:0/4029062267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a560012f0 con 0x559a560a7e80 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559a55fa9270 con 0x559a5609d5e0 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.332+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559a55ffd2b0 con 0x559a5609ccc0 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a9ec9640 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a5609d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/4029062267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 msgr2=0x559a5609d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a5609d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/4029062267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 msgr2=0x559a560a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 0x559a560a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/4029062267 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a55fa8340 con 0x559a560a7e80 2024-09-10T10:05:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a9ec9640 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a5609d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:05:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.333+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560aa270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a950 tx=0x7fb59400ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262dae446963601 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.334+0000 7fb58f7fe640 1 -- 172.21.15.45:0/4029062267 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59400cd80 con 0x559a560a7e80 2024-09-10T10:05:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.334+0000 7fb58f7fe640 1 -- 172.21.15.45:0/4029062267 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb594005500 con 0x559a560a7e80 2024-09-10T10:05:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.334+0000 7fb58f7fe640 1 -- 172.21.15.45:0/4029062267 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59400f6b0 con 0x559a560a7e80 2024-09-10T10:05:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.335+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 msgr2=0x559a560aa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.335+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560aa270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a950 tx=0x7fb59400ae10 comp rx=0 tx=0).stop 2024-09-10T10:05:22.337 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.335+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 shutdown_connections 2024-09-10T10:05:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.335+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560aa270 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.336+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 0x559a560a7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.336+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/4029062267 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a5609d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.336+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 >> 172.21.15.45:0/4029062267 conn(0x559a56099230 msgr2=0x559a56099610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.336+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 shutdown_connections 2024-09-10T10:05:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.336+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/4029062267 wait complete. 2024-09-10T10:05:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.337+0000 7fb5ae6c1740 1 -- start start 2024-09-10T10:05:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.337+0000 7fb5ae6c1740 1 Processor -- start 2024-09-10T10:05:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 0x559a560a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a560aaa50 con 0x559a560a7e80 2024-09-10T10:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559a55f8a290 con 0x559a5609d5e0 2024-09-10T10:05:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x559a560aabd0 con 0x559a5609ccc0 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5ae6c1740 10 client.0 _ll_register_callbacks cb 0x559a55fbb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.338+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.339+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560a5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48024/0 (socket says 172.21.15.45:48024) 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.339+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1963119134 learned_addr learned my addr 172.21.15.45:0/1963119134 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.339+0000 7fb5a9ec9640 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.339+0000 7fb59a7fc640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5900a2d90 0x7fb5900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.340+0000 7fb59a7fc640 1 -- 172.21.15.45:0/1865222132 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb5a00ce980 con 0x7fb59c094930 2024-09-10T10:05:22.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.340+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 0x559a560a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.341+0000 7fb5a9ec9640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5900a2d90 0x7fb5900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.341+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1963119134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 msgr2=0x559a560a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.341+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.341+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 msgr2=0x559a560a52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.341+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 0x559a560a52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.341+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1963119134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a55d14210 con 0x559a560a7e80 2024-09-10T10:05:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.342+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560a5810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a2a0 tx=0x7fb59400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77445ef4765c52c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.342+0000 7fb58cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:05:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.343+0000 7fb5a9ec9640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5900a2d90 0x7fb5900a5250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40679d0 tx=0x7fb5a4068340 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.344+0000 7fb58cff9640 1 -- 172.21.15.45:0/1963119134 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594028060 con 0x559a560a7e80 2024-09-10T10:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.344+0000 7fb58cff9640 1 -- 172.21.15.45:0/1963119134 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb594005230 con 0x559a560a7e80 2024-09-10T10:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.344+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb594005500 con 0x559a560a7e80 2024-09-10T10:05:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.344+0000 7fb58cff9640 1 -- 172.21.15.45:0/1963119134 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb59400cea0 con 0x559a560a7e80 2024-09-10T10:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.344+0000 7fb5ae6c1740 10 client.15297 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:22.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.344+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559a55de0620 con 0x559a560a7e80 2024-09-10T10:05:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.347+0000 7fb58cff9640 1 -- 172.21.15.45:0/1963119134 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb594005500 con 0x559a560a7e80 2024-09-10T10:05:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.347+0000 7fb58cff9640 1 -- 172.21.15.45:0/1963119134 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb59400f450 con 0x559a560a7e80 2024-09-10T10:05:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.348+0000 7fb5ae6c1740 10 client.15297 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.348+0000 7fb5ae6c1740 10 client.15297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.348+0000 7fb5ae6c1740 20 client.15297 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.348+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a560b81d0 0x559a560ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.348+0000 7fb5ae6c1740 4 client.15297 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:05:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.348+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a5600ea20 con 0x559a560b81d0 2024-09-10T10:05:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.349+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a560b81d0 0x559a560ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.350+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a560b81d0 0x559a560ba5c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922481366cae9eb4 server_cookie=79ada6df089b740 in_seq=0 out_seq=0 2024-09-10T10:05:22.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.350+0000 7fb58cff9640 10 client.15297 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb58cff9640 1 -- 172.21.15.45:0/1963119134 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a5600ea20 con 0x559a560b81d0 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb58cff9640 10 client.15297 handle_command_reply: tid=0 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 client.15297 shutdown 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 msgr2=0x559a560a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560a5810 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a2a0 tx=0x7fb59400cdb0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 shutdown_connections 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a560b81d0 0x559a560ba5c0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a560a7e80 0x559a560a5810 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609d5e0 0x559a560a52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb58cff9640 0 client.15297 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1963119134 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.355+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 >> 172.21.15.45:0/1963119134 conn(0x559a56099230 msgr2=0x559a5609ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.356+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 shutdown_connections 2024-09-10T10:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.356+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1963119134 wait complete. 2024-09-10T10:05:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.356+0000 7fb5ae6c1740 20 client.15297 trim_cache size 0 max 16384 2024-09-10T10:05:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.404+0000 7fb5ae6c1740 1 Processor -- start 2024-09-10T10:05:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.414+0000 7fb5ae6c1740 1 -- start start 2024-09-10T10:05:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.416+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609ccc0 0x559a56093ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.416+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.417+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a560b81d0 0x559a560bfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.417+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c145960 con 0x559a5609d5e0 2024-09-10T10:05:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.417+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c148200 con 0x559a5609ccc0 2024-09-10T10:05:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.417+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c145ff0 con 0x559a560b81d0 2024-09-10T10:05:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.420+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.420+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48026/0 (socket says 172.21.15.45:48026) 2024-09-10T10:05:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.420+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/3144060709 learned_addr learned my addr 172.21.15.45:0/3144060709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.422+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609ccc0 0x559a56093ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.424+0000 7fb5a9ec9640 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a560b81d0 0x559a560bfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.424+0000 7fb5a96c8640 1 -- 172.21.15.45:0/3144060709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a560b81d0 msgr2=0x559a560bfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.425+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a560b81d0 0x559a560bfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.425+0000 7fb5a96c8640 1 -- 172.21.15.45:0/3144060709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 msgr2=0x559a560a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.425+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.425+0000 7fb5a96c8640 1 -- 172.21.15.45:0/3144060709 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a560a3d20 con 0x559a5609ccc0 2024-09-10T10:05:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.427+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609ccc0 0x559a56093ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00cc1f0 tx=0x7fb5a00bb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ec29156749e2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.431+0000 7fb58effd640 1 -- 172.21.15.45:0/3144060709 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a009f370 con 0x559a5609ccc0 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.431+0000 7fb58effd640 1 -- 172.21.15.45:0/3144060709 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a00a3c30 con 0x559a5609ccc0 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.431+0000 7fb58effd640 1 -- 172.21.15.45:0/3144060709 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0095b30 con 0x559a5609ccc0 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.432+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/3144060709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609ccc0 msgr2=0x559a56093ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.433+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609ccc0 0x559a56093ed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00cc1f0 tx=0x7fb5a00bb040 comp rx=0 tx=0).stop 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.434+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/3144060709 shutdown_connections 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.434+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a560b81d0 0x559a560bfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.434+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.434+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/3144060709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a5609ccc0 0x559a56093ed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.434+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/3144060709 >> 172.21.15.45:0/3144060709 conn(0x559a56099230 msgr2=0x559a560a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.445+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/3144060709 shutdown_connections 2024-09-10T10:05:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.445+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/3144060709 wait complete. 2024-09-10T10:05:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.446+0000 7fb5ae6c1740 1 -- start start 2024-09-10T10:05:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.446+0000 7fb5ae6c1740 1 Processor -- start 2024-09-10T10:05:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.452+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.452+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560ab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.452+0000 7fb5ae6c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a560b81d0 0x559a560abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.452+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c1466c0 con 0x559a5609d5e0 2024-09-10T10:05:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.452+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c147b30 con 0x559a560b81d0 2024-09-10T10:05:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.452+0000 7fb5ae6c1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c146d90 con 0x559a5609ccc0 2024-09-10T10:05:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.453+0000 7fb5ae6c1740 10 client.0 _ll_register_callbacks cb 0x7fb59400a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.453+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.454+0000 7fb5a8ec7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560ab750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48028/0 (socket says 172.21.15.45:48028) 2024-09-10T10:05:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.454+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1531222544 learned_addr learned my addr 172.21.15.45:0/1531222544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.455+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.455+0000 7fb5a9ec9640 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a560b81d0 0x559a560abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.456+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1531222544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 msgr2=0x559a560b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.456+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.456+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a560b81d0 msgr2=0x559a560abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.456+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a560b81d0 0x559a560abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.460 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.456+0000 7fb5a8ec7640 1 -- 172.21.15.45:0/1531222544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a560ba930 con 0x559a5609d5e0 2024-09-10T10:05:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.466+0000 7fb5a8ec7640 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560ab750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a870 tx=0x7fb59400a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c6f6db39d8ba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.467+0000 7fb58cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:05:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.471+0000 7fb58cff9640 1 -- 172.21.15.45:0/1531222544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59400f660 con 0x559a5609d5e0 2024-09-10T10:05:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.471+0000 7fb58cff9640 1 -- 172.21.15.45:0/1531222544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb59400fa40 con 0x559a5609d5e0 2024-09-10T10:05:22.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.471+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a00a3c30 con 0x559a5609d5e0 2024-09-10T10:05:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.477+0000 7fb58cff9640 1 -- 172.21.15.45:0/1531222544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb59403b050 con 0x559a5609d5e0 2024-09-10T10:05:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.477+0000 7fb5ae6c1740 10 client.15303 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:22.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.477+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559a56122a30 con 0x559a5609d5e0 2024-09-10T10:05:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb58cff9640 1 -- 172.21.15.45:0/1531222544 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a00a3c30 con 0x559a5609d5e0 2024-09-10T10:05:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb58cff9640 1 -- 172.21.15.45:0/1531222544 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb5940035f0 con 0x559a5609d5e0 2024-09-10T10:05:22.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb5ae6c1740 10 client.15303 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb5ae6c1740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb5ae6c1740 20 client.15303 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a561230f0 0x559a56125510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb5ae6c1740 4 client.15303 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:05:22.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.484+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a56008d70 con 0x559a561230f0 2024-09-10T10:05:22.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.490+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a561230f0 0x559a56125510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.506+0000 7fb5a96c8640 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a561230f0 0x559a56125510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9ba6432b135b7b server_cookie=2b55af00df52c90a in_seq=0 out_seq=0 2024-09-10T10:05:22.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:22.508+0000 7fb58cff9640 10 client.15303 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:23.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:22 smithi162 ceph-mon[34963]: pgmap v68: 97 pgs: 97 active+clean; 18 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 686 MiB/s wr, 2.15k op/s 2024-09-10T10:05:23.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:22 smithi045 ceph-mon[31114]: pgmap v68: 97 pgs: 97 active+clean; 18 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 686 MiB/s wr, 2.15k op/s 2024-09-10T10:05:23.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:22 smithi047 ceph-mon[35244]: pgmap v68: 97 pgs: 97 active+clean; 18 GiB data, 79 GiB used, 994 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 686 MiB/s wr, 2.15k op/s 2024-09-10T10:05:25.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:24 smithi162 ceph-mon[34963]: pgmap v69: 97 pgs: 97 active+clean; 19 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.0 GiB/s wr, 3.01k op/s 2024-09-10T10:05:25.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:24 smithi045 ceph-mon[31114]: pgmap v69: 97 pgs: 97 active+clean; 19 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.0 GiB/s wr, 3.01k op/s 2024-09-10T10:05:25.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:24 smithi047 ceph-mon[35244]: pgmap v69: 97 pgs: 97 active+clean; 19 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.0 GiB/s wr, 3.01k op/s 2024-09-10T10:05:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.500+0000 7fb58cff9640 1 -- 172.21.15.45:0/1531222544 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a56008d70 con 0x559a561230f0 2024-09-10T10:05:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.500+0000 7fb58cff9640 10 client.15303 handle_command_reply: tid=0 2024-09-10T10:05:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.500+0000 7fb5ae6c1740 1 client.15303 shutdown 2024-09-10T10:05:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.500+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 msgr2=0x559a560ab750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560ab750 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb59400a870 tx=0x7fb59400a690 comp rx=0 tx=0).stop 2024-09-10T10:05:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 shutdown_connections 2024-09-10T10:05:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x559a561230f0 0x559a56125510 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559a560b81d0 0x559a560abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a5609d5e0 0x559a560ab750 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 --2- 172.21.15.45:0/1531222544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x559a5609ccc0 0x559a560b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 >> 172.21.15.45:0/1531222544 conn(0x559a56099230 msgr2=0x559a56097a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb58cff9640 0 client.15303 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.501+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 shutdown_connections 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.502+0000 7fb5ae6c1740 1 -- 172.21.15.45:0/1531222544 wait complete. 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.502+0000 7fb5ae6c1740 20 client.15303 trim_cache size 0 max 16384 2024-09-10T10:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:05:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.508+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5900a2d90 msgr2=0x7fb5900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.508+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5900a2d90 0x7fb5900a5250 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40679d0 tx=0x7fb5a4068340 comp rx=0 tx=0).stop 2024-09-10T10:05:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.508+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c094930 msgr2=0x7fb59c12be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.508+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c094930 0x7fb59c12be70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000b4a0 tx=0x7fb5a000b960 comp rx=0 tx=0).stop 2024-09-10T10:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.513+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 shutdown_connections 2024-09-10T10:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.513+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb59c09a5b0 0x7fb59c130730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.513+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5900a2d90 0x7fb5900a5250 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.513+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb59c0952d0 0x7fb59c12c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.513+0000 7fb5aa6ca640 1 --2- 172.21.15.45:0/1865222132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb59c094930 0x7fb59c12be70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.513+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 >> 172.21.15.45:0/1865222132 conn(0x7fb59c00a220 msgr2=0x7fb59c093be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.515+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 shutdown_connections 2024-09-10T10:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:25.515+0000 7fb5aa6ca640 1 -- 172.21.15.45:0/1865222132 wait complete. 2024-09-10T10:05:27.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:26 smithi162 ceph-mon[34963]: pgmap v70: 97 pgs: 97 active+clean; 19 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 699 MiB/s wr, 1.28k op/s 2024-09-10T10:05:27.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:26 smithi047 ceph-mon[35244]: pgmap v70: 97 pgs: 97 active+clean; 19 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 699 MiB/s wr, 1.28k op/s 2024-09-10T10:05:27.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:26 smithi045 ceph-mon[31114]: pgmap v70: 97 pgs: 97 active+clean; 19 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 699 MiB/s wr, 1.28k op/s 2024-09-10T10:05:29.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:28 smithi162 ceph-mon[34963]: pgmap v71: 97 pgs: 97 active+clean; 19 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 813 MiB/s wr, 1.43k op/s 2024-09-10T10:05:29.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:28 smithi045 ceph-mon[31114]: pgmap v71: 97 pgs: 97 active+clean; 19 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 813 MiB/s wr, 1.43k op/s 2024-09-10T10:05:29.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:28 smithi047 ceph-mon[35244]: pgmap v71: 97 pgs: 97 active+clean; 19 GiB data, 85 GiB used, 988 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 813 MiB/s wr, 1.43k op/s 2024-09-10T10:05:31.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:30 smithi162 ceph-mon[34963]: pgmap v72: 97 pgs: 97 active+clean; 20 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 950 MiB/s wr, 1.62k op/s 2024-09-10T10:05:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:30 smithi045 ceph-mon[31114]: pgmap v72: 97 pgs: 97 active+clean; 20 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 950 MiB/s wr, 1.62k op/s 2024-09-10T10:05:31.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:30 smithi047 ceph-mon[35244]: pgmap v72: 97 pgs: 97 active+clean; 20 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 950 MiB/s wr, 1.62k op/s 2024-09-10T10:05:33.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:33 smithi162 ceph-mon[34963]: pgmap v73: 97 pgs: 97 active+clean; 20 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 717 MiB/s wr, 1.34k op/s 2024-09-10T10:05:33.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:33 smithi045 ceph-mon[31114]: pgmap v73: 97 pgs: 97 active+clean; 20 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 717 MiB/s wr, 1.34k op/s 2024-09-10T10:05:33.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:33 smithi047 ceph-mon[35244]: pgmap v73: 97 pgs: 97 active+clean; 20 GiB data, 87 GiB used, 986 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 717 MiB/s wr, 1.34k op/s 2024-09-10T10:05:35.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:35 smithi162 ceph-mon[34963]: pgmap v74: 97 pgs: 97 active+clean; 20 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 GiB/s wr, 2.17k op/s 2024-09-10T10:05:35.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:35 smithi045 ceph-mon[31114]: pgmap v74: 97 pgs: 97 active+clean; 20 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 GiB/s wr, 2.17k op/s 2024-09-10T10:05:35.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:35 smithi047 ceph-mon[35244]: pgmap v74: 97 pgs: 97 active+clean; 20 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 GiB/s wr, 2.17k op/s 2024-09-10T10:05:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:37 smithi162 ceph-mon[34963]: pgmap v75: 97 pgs: 97 active+clean; 20 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 691 MiB/s wr, 1.32k op/s 2024-09-10T10:05:37.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:37 smithi045 ceph-mon[31114]: pgmap v75: 97 pgs: 97 active+clean; 20 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 691 MiB/s wr, 1.32k op/s 2024-09-10T10:05:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:37 smithi047 ceph-mon[35244]: pgmap v75: 97 pgs: 97 active+clean; 20 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 691 MiB/s wr, 1.32k op/s 2024-09-10T10:05:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.826+0000 7fb76c711640 1 -- 172.21.15.45:0/1825445148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb76411b140 msgr2=0x7fb76411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.826+0000 7fb76c711640 1 --2- 172.21.15.45:0/1825445148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb76411b140 0x7fb76411d530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb760009890 tx=0x7fb76002a380 comp rx=0 tx=0).stop 2024-09-10T10:05:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.830+0000 7fb76c711640 1 -- 172.21.15.45:0/1825445148 shutdown_connections 2024-09-10T10:05:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.830+0000 7fb76c711640 1 --2- 172.21.15.45:0/1825445148 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb76411b140 0x7fb76411d530 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.830+0000 7fb76c711640 1 --2- 172.21.15.45:0/1825445148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7641095d0 0x7fb764109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.830+0000 7fb76c711640 1 --2- 172.21.15.45:0/1825445148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7641091f0 0x7fb764110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.830+0000 7fb76c711640 1 -- 172.21.15.45:0/1825445148 >> 172.21.15.45:0/1825445148 conn(0x7fb76406ce00 msgr2=0x7fb76406d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.831+0000 7fb76c711640 1 -- 172.21.15.45:0/1825445148 shutdown_connections 2024-09-10T10:05:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.831+0000 7fb76c711640 1 -- 172.21.15.45:0/1825445148 wait complete. 2024-09-10T10:05:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 Processor -- start 2024-09-10T10:05:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 -- start start 2024-09-10T10:05:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7641091f0 0x7fb76411a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7641095d0 0x7fb7641148e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb76411b140 0x7fb764114e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb764111370 con 0x7fb7641091f0 2024-09-10T10:05:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641111f0 con 0x7fb76411b140 2024-09-10T10:05:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.833+0000 7fb76c711640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb7641114f0 con 0x7fb7641095d0 2024-09-10T10:05:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.834+0000 7fb769cac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7641095d0 0x7fb7641148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.835+0000 7fb769cac640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7641095d0 0x7fb7641148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:44662/0 (socket says 172.21.15.45:44662) 2024-09-10T10:05:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.835+0000 7fb769cac640 1 -- 172.21.15.45:0/914556394 learned_addr learned my addr 172.21.15.45:0/914556394 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.836+0000 7fb76acae640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb76411b140 0x7fb764114e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.836+0000 7fb769cac640 1 -- 172.21.15.45:0/914556394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb76411b140 msgr2=0x7fb764114e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.836+0000 7fb769cac640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb76411b140 0x7fb764114e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.836+0000 7fb769cac640 1 -- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7641091f0 msgr2=0x7fb76411a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.836+0000 7fb769cac640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7641091f0 0x7fb76411a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.836+0000 7fb769cac640 1 -- 172.21.15.45:0/914556394 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb764115550 con 0x7fb7641095d0 2024-09-10T10:05:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.837+0000 7fb769cac640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7641095d0 0x7fb7641148e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00c420 tx=0x7fb75c00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:38.844 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.842+0000 7fb75b7fe640 1 -- 172.21.15.45:0/914556394 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c004b20 con 0x7fb7641095d0 2024-09-10T10:05:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.843+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7641bc000 con 0x7fb7641095d0 2024-09-10T10:05:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.843+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7641bc4c0 con 0x7fb7641095d0 2024-09-10T10:05:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.849+0000 7fb76f4ad740 1 Processor -- start 2024-09-10T10:05:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.850+0000 7fb75b7fe640 1 -- 172.21.15.45:0/914556394 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75c003e50 con 0x7fb7641095d0 2024-09-10T10:05:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.850+0000 7fb75b7fe640 1 -- 172.21.15.45:0/914556394 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00b630 con 0x7fb7641095d0 2024-09-10T10:05:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.850+0000 7fb76f4ad740 1 -- start start 2024-09-10T10:05:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.868+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939aaf0 0x55b7e939aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.871+0000 7fb76acae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939aaf0 0x55b7e939aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.871+0000 7fb76acae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939aaf0 0x55b7e939aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:44672/0 (socket says 172.21.15.45:44672) 2024-09-10T10:05:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.876+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.876+0000 7fb76acae640 1 -- 172.21.15.45:0/915355701 learned_addr learned my addr 172.21.15.45:0/915355701 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.876+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.876+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b7e92ff2f0 con 0x55b7e939b410 2024-09-10T10:05:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.876+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b7e92a7270 con 0x55b7e93a5cb0 2024-09-10T10:05:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.876+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b7e92fb2b0 con 0x55b7e939aaf0 2024-09-10T10:05:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.878+0000 7fb75b7fe640 1 -- 172.21.15.45:0/914556394 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb75c01d020 con 0x7fb7641095d0 2024-09-10T10:05:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.879+0000 7fb769cac640 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.880+0000 7fb75b7fe640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb7500a2f80 0x7fb7500a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.880+0000 7fb76acae640 1 -- 172.21.15.45:0/915355701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a5cb0 msgr2=0x55b7e93a80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.880+0000 7fb76acae640 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.881+0000 7fb76acae640 1 -- 172.21.15.45:0/915355701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 msgr2=0x55b7e93a5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.881+0000 7fb76acae640 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.881+0000 7fb76acae640 1 -- 172.21.15.45:0/915355701 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e92a6340 con 0x55b7e939aaf0 2024-09-10T10:05:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.882+0000 7fb75b7fe640 1 -- 172.21.15.45:0/914556394 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb75c0cf360 con 0x7fb7641095d0 2024-09-10T10:05:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.886+0000 7fb76acae640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb7500a2f80 0x7fb7500a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.886+0000 7fb76acae640 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939aaf0 0x55b7e939aed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb760005b60 tx=0x7fb760009440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=859fb5e7da426006 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.891+0000 7fb758ff9640 1 -- 172.21.15.45:0/915355701 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760037040 con 0x55b7e939aaf0 2024-09-10T10:05:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.891+0000 7fb758ff9640 1 -- 172.21.15.45:0/915355701 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb760043030 con 0x55b7e939aaf0 2024-09-10T10:05:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.894+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939aaf0 msgr2=0x55b7e939aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.894+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939aaf0 0x55b7e939aed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb760005b60 tx=0x7fb760009440 comp rx=0 tx=0).stop 2024-09-10T10:05:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.895+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 shutdown_connections 2024-09-10T10:05:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.895+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.895+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.895+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/915355701 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939aaf0 0x55b7e939aed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.895+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 >> 172.21.15.45:0/915355701 conn(0x55b7e9397060 msgr2=0x55b7e9397440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.895+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 shutdown_connections 2024-09-10T10:05:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.898+0000 7fb76f4ad740 1 -- 172.21.15.45:0/915355701 wait complete. 2024-09-10T10:05:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.899+0000 7fb76f4ad740 1 -- start start 2024-09-10T10:05:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.899+0000 7fb76f4ad740 1 Processor -- start 2024-09-10T10:05:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.900+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939b410 0x55b7e93a5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.900+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.900+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.900+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b7e93a88d0 con 0x55b7e93a5cb0 2024-09-10T10:05:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.900+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b7e9288290 con 0x55b7e93a2ec0 2024-09-10T10:05:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.900+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b7e93a8a50 con 0x55b7e939b410 2024-09-10T10:05:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.900+0000 7fb76f4ad740 10 client.0 _ll_register_callbacks cb 0x55b7e92b9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.903+0000 7fb76a4ad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939b410 0x55b7e93a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.903+0000 7fb769cac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.904+0000 7fb769cac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54484/0 (socket says 172.21.15.45:54484) 2024-09-10T10:05:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.904+0000 7fb769cac640 1 -- 172.21.15.45:0/3910808848 learned_addr learned my addr 172.21.15.45:0/3910808848 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.907+0000 7fb76acae640 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.907+0000 7fb76a4ad640 1 -- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 msgr2=0x55b7e93a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.907+0000 7fb76a4ad640 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.907+0000 7fb76a4ad640 1 -- 172.21.15.45:0/3910808848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e93a5cb0 msgr2=0x55b7e93a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.907+0000 7fb76a4ad640 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.907+0000 7fb76a4ad640 1 -- 172.21.15.45:0/3910808848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e9012210 con 0x55b7e939b410 2024-09-10T10:05:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.908+0000 7fb76a4ad640 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939b410 0x55b7e93a5690 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb754009e00 tx=0x7fb75400b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c15ef0a97ad8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.909+0000 7fb74dffb640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:05:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.910+0000 7fb74dffb640 1 -- 172.21.15.45:0/3910808848 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754015020 con 0x55b7e939b410 2024-09-10T10:05:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.914+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7e930ca20 con 0x55b7e939b410 2024-09-10T10:05:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.914+0000 7fb74dffb640 1 -- 172.21.15.45:0/3910808848 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb754011420 con 0x55b7e939b410 2024-09-10T10:05:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.915+0000 7fb74dffb640 1 -- 172.21.15.45:0/3910808848 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb754020050 con 0x55b7e939b410 2024-09-10T10:05:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.915+0000 7fb76f4ad740 10 client.24928 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:38.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.915+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b7e90de620 con 0x55b7e939b410 2024-09-10T10:05:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.918+0000 7fb74dffb640 1 -- 172.21.15.45:0/3910808848 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b7e930ca20 con 0x55b7e939b410 2024-09-10T10:05:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.918+0000 7fb76acae640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb7500a2f80 0x7fb7500a5440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb760030490 tx=0x7fb760009440 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.919+0000 7fb74dffb640 1 -- 172.21.15.45:0/3910808848 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb7540198c0 con 0x55b7e939b410 2024-09-10T10:05:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.919+0000 7fb76f4ad740 10 client.24928 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.919+0000 7fb76f4ad740 10 client.24928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.919+0000 7fb76f4ad740 20 client.24928 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.919+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e93b6310 0x55b7e93b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.919+0000 7fb76f4ad740 4 client.24928 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:05:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.919+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7e9306d70 con 0x55b7e93b6310 2024-09-10T10:05:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.921+0000 7fb769cac640 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e93b6310 0x55b7e93b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.936+0000 7fb74dffb640 10 client.24928 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.936+0000 7fb769cac640 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e93b6310 0x55b7e93b8730 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3511a5e15efd116b server_cookie=c86322f2278d4dfd in_seq=0 out_seq=0 2024-09-10T10:05:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.941+0000 7fb74dffb640 1 -- 172.21.15.45:0/3910808848 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7e9306d70 con 0x55b7e93b6310 2024-09-10T10:05:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.941+0000 7fb74dffb640 10 client.24928 handle_command_reply: tid=0 2024-09-10T10:05:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.941+0000 7fb76f4ad740 1 client.24928 shutdown 2024-09-10T10:05:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.947+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939b410 msgr2=0x55b7e93a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.947+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939b410 0x55b7e93a5690 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb754009e00 tx=0x7fb75400b4b0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.952+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 shutdown_connections 2024-09-10T10:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.952+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e93b6310 0x55b7e93b8730 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.952+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.952+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e93a5cb0 0x55b7e93a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.952+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/3910808848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e939b410 0x55b7e93a5690 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.952+0000 7fb74dffb640 0 client.24928 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.952+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 >> 172.21.15.45:0/3910808848 conn(0x55b7e9397060 msgr2=0x55b7e939c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.953+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 shutdown_connections 2024-09-10T10:05:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.956+0000 7fb76f4ad740 1 -- 172.21.15.45:0/3910808848 wait complete. 2024-09-10T10:05:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:38.956+0000 7fb76f4ad740 20 client.24928 trim_cache size 0 max 16384 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.028+0000 7fb76f4ad740 1 Processor -- start 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.028+0000 7fb76f4ad740 1 -- start start 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.029+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.029+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.029+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e939c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.029+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb74800c740 con 0x55b7e939b410 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.029+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb74800ef10 con 0x55b7e93a2ec0 2024-09-10T10:05:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.029+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb74800cd00 con 0x55b7e93a5cb0 2024-09-10T10:05:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.030+0000 7fb769cac640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.030+0000 7fb76acae640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e939c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.031+0000 7fb76a4ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.031+0000 7fb76a4ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54486/0 (socket says 172.21.15.45:54486) 2024-09-10T10:05:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.031+0000 7fb76a4ad640 1 -- 172.21.15.45:0/648855518 learned_addr learned my addr 172.21.15.45:0/648855518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.033+0000 7fb76acae640 1 -- 172.21.15.45:0/648855518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 msgr2=0x55b7e93a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.033+0000 7fb76acae640 1 --2- 172.21.15.45:0/648855518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.033+0000 7fb76acae640 1 -- 172.21.15.45:0/648855518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 msgr2=0x55b7e93a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.033+0000 7fb76acae640 1 --2- 172.21.15.45:0/648855518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.033+0000 7fb76acae640 1 -- 172.21.15.45:0/648855518 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7e939a870 con 0x55b7e93a5cb0 2024-09-10T10:05:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.037+0000 7fb76acae640 1 --2- 172.21.15.45:0/648855518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e939c550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb76002a970 tx=0x7fb76002c070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a7455436d42ef45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.040+0000 7fb74ffff640 1 -- 172.21.15.45:0/648855518 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76004e030 con 0x55b7e93a5cb0 2024-09-10T10:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.040+0000 7fb74ffff640 1 -- 172.21.15.45:0/648855518 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb760058070 con 0x55b7e93a5cb0 2024-09-10T10:05:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.040+0000 7fb74ffff640 1 -- 172.21.15.45:0/648855518 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760042040 con 0x55b7e93a5cb0 2024-09-10T10:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 -- 172.21.15.45:0/648855518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 msgr2=0x55b7e939c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/648855518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e939c550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb76002a970 tx=0x7fb76002c070 comp rx=0 tx=0).stop 2024-09-10T10:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 -- 172.21.15.45:0/648855518 shutdown_connections 2024-09-10T10:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/648855518 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e939c550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/648855518 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/648855518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 -- 172.21.15.45:0/648855518 >> 172.21.15.45:0/648855518 conn(0x55b7e9397060 msgr2=0x55b7e939dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 -- 172.21.15.45:0/648855518 shutdown_connections 2024-09-10T10:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.042+0000 7fb76f4ad740 1 -- 172.21.15.45:0/648855518 wait complete. 2024-09-10T10:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.043+0000 7fb76f4ad740 1 -- start start 2024-09-10T10:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.043+0000 7fb76f4ad740 1 Processor -- start 2024-09-10T10:05:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.044+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.044+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93be280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.044+0000 7fb76f4ad740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e93be7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.044+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb74800d3d0 con 0x55b7e939b410 2024-09-10T10:05:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.044+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb74800e840 con 0x55b7e93a2ec0 2024-09-10T10:05:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.044+0000 7fb76f4ad740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb74800daa0 con 0x55b7e93a5cb0 2024-09-10T10:05:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.045+0000 7fb76f4ad740 10 client.0 _ll_register_callbacks cb 0x7fb754006f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.045+0000 7fb76a4ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.045+0000 7fb76a4ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93bdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54496/0 (socket says 172.21.15.45:54496) 2024-09-10T10:05:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.045+0000 7fb76a4ad640 1 -- 172.21.15.45:0/4205607445 learned_addr learned my addr 172.21.15.45:0/4205607445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.045+0000 7fb769cac640 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93be280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.046+0000 7fb76acae640 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e93be7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.046+0000 7fb76a4ad640 1 -- 172.21.15.45:0/4205607445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 msgr2=0x55b7e93be7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.046+0000 7fb76a4ad640 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e93be7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.046+0000 7fb76a4ad640 1 -- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 msgr2=0x55b7e93be280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.046+0000 7fb76a4ad640 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93be280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.046+0000 7fb76a4ad640 1 -- 172.21.15.45:0/4205607445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7e93ac4c0 con 0x55b7e939b410 2024-09-10T10:05:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.046+0000 7fb76a4ad640 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93bdd40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb75400bfe0 tx=0x7fb754004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c78fa34a303de5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.048+0000 7fb74dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:05:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.048+0000 7fb74dffb640 1 -- 172.21.15.45:0/4205607445 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754028070 con 0x55b7e939b410 2024-09-10T10:05:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.049+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb760058070 con 0x55b7e939b410 2024-09-10T10:05:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.049+0000 7fb74dffb640 1 -- 172.21.15.45:0/4205607445 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7fb7540195a0 con 0x55b7e939b410 2024-09-10T10:05:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.049+0000 7fb74dffb640 1 -- 172.21.15.45:0/4205607445 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fb754015040 con 0x55b7e939b410 2024-09-10T10:05:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.049+0000 7fb76f4ad740 10 client.15312 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.049+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b7e94213f0 con 0x55b7e939b410 2024-09-10T10:05:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.050+0000 7fb74dffb640 1 -- 172.21.15.45:0/4205607445 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb760058070 con 0x55b7e939b410 2024-09-10T10:05:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.051+0000 7fb74dffb640 1 -- 172.21.15.45:0/4205607445 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7fb754010d40 con 0x55b7e939b410 2024-09-10T10:05:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.051+0000 7fb76f4ad740 10 client.15312 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.051+0000 7fb76f4ad740 10 client.15312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.051+0000 7fb76f4ad740 20 client.15312 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.051+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e9421a90 0x55b7e9423ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.051+0000 7fb76f4ad740 4 client.15312 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:05:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.051+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7e939b260 con 0x55b7e9421a90 2024-09-10T10:05:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.052+0000 7fb769cac640 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e9421a90 0x55b7e9423ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.054+0000 7fb769cac640 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e9421a90 0x55b7e9423ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71fa2dc2f7bdc086 server_cookie=ad817de8eadf5f84 in_seq=0 out_seq=0 2024-09-10T10:05:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:39.054+0000 7fb74dffb640 10 client.15312 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:39.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:39 smithi162 ceph-mon[34963]: pgmap v76: 97 pgs: 97 active+clean; 20 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 797 MiB/s wr, 1.42k op/s 2024-09-10T10:05:39.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:39 smithi045 ceph-mon[31114]: pgmap v76: 97 pgs: 97 active+clean; 20 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 797 MiB/s wr, 1.42k op/s 2024-09-10T10:05:39.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:39 smithi047 ceph-mon[35244]: pgmap v76: 97 pgs: 97 active+clean; 20 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 797 MiB/s wr, 1.42k op/s 2024-09-10T10:05:41.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:41 smithi162 ceph-mon[34963]: pgmap v77: 97 pgs: 97 active+clean; 20 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 943 MiB/s wr, 1.53k op/s 2024-09-10T10:05:41.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:41 smithi045 ceph-mon[31114]: pgmap v77: 97 pgs: 97 active+clean; 20 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 943 MiB/s wr, 1.53k op/s 2024-09-10T10:05:41.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:41 smithi047 ceph-mon[35244]: pgmap v77: 97 pgs: 97 active+clean; 20 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 943 MiB/s wr, 1.53k op/s 2024-09-10T10:05:43.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:43 smithi162 ceph-mon[34963]: pgmap v78: 97 pgs: 97 active+clean; 20 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 699 MiB/s wr, 1.19k op/s 2024-09-10T10:05:43.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:43 smithi045 ceph-mon[31114]: pgmap v78: 97 pgs: 97 active+clean; 20 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 699 MiB/s wr, 1.19k op/s 2024-09-10T10:05:43.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:43 smithi047 ceph-mon[35244]: pgmap v78: 97 pgs: 97 active+clean; 20 GiB data, 96 GiB used, 977 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 699 MiB/s wr, 1.19k op/s 2024-09-10T10:05:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.772+0000 7fb74dffb640 1 -- 172.21.15.45:0/4205607445 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7e939b260 con 0x55b7e9421a90 2024-09-10T10:05:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.772+0000 7fb74dffb640 10 client.15312 handle_command_reply: tid=0 2024-09-10T10:05:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.772+0000 7fb76f4ad740 1 client.15312 shutdown 2024-09-10T10:05:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.774+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 msgr2=0x55b7e93bdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.774+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93bdd40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb75400bfe0 tx=0x7fb754004880 comp rx=0 tx=0).stop 2024-09-10T10:05:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.774+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 shutdown_connections 2024-09-10T10:05:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.774+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b7e9421a90 0x55b7e9423ed0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b7e93a5cb0 0x55b7e93be7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b7e93a2ec0 0x55b7e93be280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb76f4ad740 1 --2- 172.21.15.45:0/4205607445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b7e939b410 0x55b7e93bdd40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 >> 172.21.15.45:0/4205607445 conn(0x55b7e9397060 msgr2=0x55b7e939c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb74dffb640 0 client.15312 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 shutdown_connections 2024-09-10T10:05:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb76f4ad740 1 -- 172.21.15.45:0/4205607445 wait complete. 2024-09-10T10:05:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.775+0000 7fb76f4ad740 20 client.15312 trim_cache size 0 max 16384 2024-09-10T10:05:44.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:05:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:05:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:05:44.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:05:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.782+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb7500a2f80 msgr2=0x7fb7500a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.782+0000 7fb76c711640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb7500a2f80 0x7fb7500a5440 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb760030490 tx=0x7fb760009440 comp rx=0 tx=0).stop 2024-09-10T10:05:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.782+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7641095d0 msgr2=0x7fb7641148e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.782+0000 7fb76c711640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7641095d0 0x7fb7641148e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00c420 tx=0x7fb75c00c8e0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.783+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 shutdown_connections 2024-09-10T10:05:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.783+0000 7fb76c711640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb7500a2f80 0x7fb7500a5440 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.783+0000 7fb76c711640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb76411b140 0x7fb764114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.783+0000 7fb76c711640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb7641095d0 0x7fb7641148e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.783+0000 7fb76c711640 1 --2- 172.21.15.45:0/914556394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7641091f0 0x7fb76411a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.783+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 >> 172.21.15.45:0/914556394 conn(0x7fb76406ce00 msgr2=0x7fb764070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.786+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 shutdown_connections 2024-09-10T10:05:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:44.787+0000 7fb76c711640 1 -- 172.21.15.45:0/914556394 wait complete. 2024-09-10T10:05:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:45 smithi162 ceph-mon[34963]: pgmap v79: 97 pgs: 97 active+clean; 20 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 GiB/s wr, 1.83k op/s 2024-09-10T10:05:45.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:45 smithi045 ceph-mon[31114]: pgmap v79: 97 pgs: 97 active+clean; 20 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 GiB/s wr, 1.83k op/s 2024-09-10T10:05:45.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:45 smithi047 ceph-mon[35244]: pgmap v79: 97 pgs: 97 active+clean; 20 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 GiB/s wr, 1.83k op/s 2024-09-10T10:05:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:47 smithi162 ceph-mon[34963]: pgmap v80: 97 pgs: 97 active+clean; 20 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 734 MiB/s wr, 999 op/s 2024-09-10T10:05:47.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:47 smithi045 ceph-mon[31114]: pgmap v80: 97 pgs: 97 active+clean; 20 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 734 MiB/s wr, 999 op/s 2024-09-10T10:05:47.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:47 smithi047 ceph-mon[35244]: pgmap v80: 97 pgs: 97 active+clean; 20 GiB data, 100 GiB used, 973 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 734 MiB/s wr, 999 op/s 2024-09-10T10:05:49.896 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:49 smithi045 ceph-mon[31114]: pgmap v81: 97 pgs: 97 active+clean; 20 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 832 MiB/s wr, 1.18k op/s 2024-09-10T10:05:49.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:49 smithi047 ceph-mon[35244]: pgmap v81: 97 pgs: 97 active+clean; 20 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 832 MiB/s wr, 1.18k op/s 2024-09-10T10:05:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:49 smithi162 ceph-mon[34963]: pgmap v81: 97 pgs: 97 active+clean; 20 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 832 MiB/s wr, 1.18k op/s 2024-09-10T10:05:51.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:50 smithi162 ceph-mon[34963]: pgmap v82: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 968 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 976 MiB/s wr, 1.50k op/s 2024-09-10T10:05:51.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:50 smithi045 ceph-mon[31114]: pgmap v82: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 968 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 976 MiB/s wr, 1.50k op/s 2024-09-10T10:05:51.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:50 smithi047 ceph-mon[35244]: pgmap v82: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 968 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 976 MiB/s wr, 1.50k op/s 2024-09-10T10:05:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:52 smithi162 ceph-mon[34963]: pgmap v83: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 968 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 716 MiB/s wr, 1.24k op/s 2024-09-10T10:05:53.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:52 smithi045 ceph-mon[31114]: pgmap v83: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 968 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 716 MiB/s wr, 1.24k op/s 2024-09-10T10:05:53.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:52 smithi047 ceph-mon[35244]: pgmap v83: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 968 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 716 MiB/s wr, 1.24k op/s 2024-09-10T10:05:55.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:54 smithi162 ceph-mon[34963]: pgmap v84: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.0 GiB/s wr, 1.68k op/s 2024-09-10T10:05:55.152 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:54 smithi045 ceph-mon[31114]: pgmap v84: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.0 GiB/s wr, 1.68k op/s 2024-09-10T10:05:55.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:54 smithi047 ceph-mon[35244]: pgmap v84: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.0 GiB/s wr, 1.68k op/s 2024-09-10T10:05:57.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:57 smithi162 ceph-mon[34963]: pgmap v85: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.12k op/s 2024-09-10T10:05:57.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:57 smithi045 ceph-mon[31114]: pgmap v85: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.12k op/s 2024-09-10T10:05:57.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:57 smithi047 ceph-mon[35244]: pgmap v85: 97 pgs: 97 active+clean; 20 GiB data, 104 GiB used, 969 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.12k op/s 2024-09-10T10:05:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 -- 172.21.15.45:0/237028070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109e10 msgr2=0x7f3700073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 --2- 172.21.15.45:0/237028070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109e10 0x7f3700073f10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009d50 tx=0x7f36f002a840 comp rx=0 tx=0).stop 2024-09-10T10:05:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 -- 172.21.15.45:0/237028070 shutdown_connections 2024-09-10T10:05:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 --2- 172.21.15.45:0/237028070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3700074620 0x7f370007b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 --2- 172.21.15.45:0/237028070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109e10 0x7f3700073f10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 --2- 172.21.15.45:0/237028070 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3700109460 0x7f3700109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 -- 172.21.15.45:0/237028070 >> 172.21.15.45:0/237028070 conn(0x7f370006cbf0 msgr2=0x7f370006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 -- 172.21.15.45:0/237028070 shutdown_connections 2024-09-10T10:05:58.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.146+0000 7f36fffff640 1 -- 172.21.15.45:0/237028070 wait complete. 2024-09-10T10:05:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 Processor -- start 2024-09-10T10:05:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 -- start start 2024-09-10T10:05:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3700074620 0x7f37000847c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109460 0x7f3700084d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370007f8d0 0x7f370007fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f370007e4d0 con 0x7f3700074620 2024-09-10T10:05:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f370007e350 con 0x7f3700109460 2024-09-10T10:05:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.148+0000 7f36fffff640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f370007e650 con 0x7f370007f8d0 2024-09-10T10:05:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.149+0000 7f36fe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109460 0x7f3700084d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.149+0000 7f36fe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109460 0x7f3700084d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:50304/0 (socket says 172.21.15.45:50304) 2024-09-10T10:05:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.149+0000 7f36fe7fc640 1 -- 172.21.15.45:0/3581970433 learned_addr learned my addr 172.21.15.45:0/3581970433 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.150+0000 7f36feffd640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3700074620 0x7f37000847c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.150+0000 7f36ff7fe640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370007f8d0 0x7f370007fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.150+0000 7f36feffd640 1 -- 172.21.15.45:0/3581970433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370007f8d0 msgr2=0x7f370007fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.150+0000 7f36feffd640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370007f8d0 0x7f370007fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.150+0000 7f36feffd640 1 -- 172.21.15.45:0/3581970433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109460 msgr2=0x7f3700084d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.150+0000 7f36feffd640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109460 0x7f3700084d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.151+0000 7f36feffd640 1 -- 172.21.15.45:0/3581970433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37000805f0 con 0x7f3700074620 2024-09-10T10:05:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.151+0000 7f36feffd640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3700074620 0x7f37000847c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f36f400e9f0 tx=0x7f36f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.152+0000 7f36e7fff640 1 -- 172.21.15.45:0/3581970433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f400ccc0 con 0x7f3700074620 2024-09-10T10:05:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.158+0000 7f36e7fff640 1 -- 172.21.15.45:0/3581970433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f4010910 con 0x7f3700074620 2024-09-10T10:05:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.158+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3700137300 con 0x7f3700074620 2024-09-10T10:05:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.158+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37001378a0 con 0x7f3700074620 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.158+0000 7f36e7fff640 1 -- 172.21.15.45:0/3581970433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4018960 con 0x7f3700074620 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 Processor -- start 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 -- start start 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff585e0 0x55ac1ff62940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff62e80 0x55ac1ff65270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac1febc2f0 con 0x55ac1ff57cc0 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ac1fe64270 con 0x55ac1ff585e0 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.163+0000 7f3708475740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ac1feb82b0 con 0x55ac1ff62e80 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.164+0000 7f36fe7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff62e80 0x55ac1ff65270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.164+0000 7f36fe7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff62e80 0x55ac1ff65270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:55084/0 (socket says 172.21.15.45:55084) 2024-09-10T10:05:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.164+0000 7f36fe7fc640 1 -- 172.21.15.45:0/328366598 learned_addr learned my addr 172.21.15.45:0/328366598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.164+0000 7f36feffd640 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff585e0 0x55ac1ff62940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.166+0000 7f36feffd640 1 -- 172.21.15.45:0/328366598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff62e80 msgr2=0x55ac1ff65270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.166+0000 7f36feffd640 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff62e80 0x55ac1ff65270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.166+0000 7f36feffd640 1 -- 172.21.15.45:0/328366598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 msgr2=0x55ac1ff580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.166+0000 7f36feffd640 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.166+0000 7f36feffd640 1 -- 172.21.15.45:0/328366598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1fe63340 con 0x55ac1ff585e0 2024-09-10T10:05:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.167+0000 7f36feffd640 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff585e0 0x55ac1ff62940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3700137ac0 tx=0x7f36f4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c80941a88f896d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.170+0000 7f36e7fff640 1 -- 172.21.15.45:0/3581970433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f36f4032070 con 0x7f3700074620 2024-09-10T10:05:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.171+0000 7f36e7fff640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f36e80a2e30 0x7f36e80a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.172+0000 7f36e7fff640 1 -- 172.21.15.45:0/3581970433 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f36f40d22d0 con 0x7f3700074620 2024-09-10T10:05:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.178+0000 7f36ff7fe640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f36e80a2e30 0x7f36e80a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.178+0000 7f36e57fa640 1 -- 172.21.15.45:0/328366598 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f40d2840 con 0x55ac1ff585e0 2024-09-10T10:05:58.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.179+0000 7f36e57fa640 1 -- 172.21.15.45:0/328366598 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f40d30b0 con 0x55ac1ff585e0 2024-09-10T10:05:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.180+0000 7f3708475740 1 -- 172.21.15.45:0/328366598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff585e0 msgr2=0x55ac1ff62940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.180+0000 7f3708475740 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff585e0 0x55ac1ff62940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3700137ac0 tx=0x7f36f4017000 comp rx=0 tx=0).stop 2024-09-10T10:05:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.180+0000 7f3708475740 1 -- 172.21.15.45:0/328366598 shutdown_connections 2024-09-10T10:05:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.180+0000 7f3708475740 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff62e80 0x55ac1ff65270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.180+0000 7f3708475740 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff585e0 0x55ac1ff62940 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.180+0000 7f3708475740 1 --2- 172.21.15.45:0/328366598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.180+0000 7f3708475740 1 -- 172.21.15.45:0/328366598 >> 172.21.15.45:0/328366598 conn(0x55ac1ff54230 msgr2=0x55ac1ff54610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.181+0000 7f3708475740 1 -- 172.21.15.45:0/328366598 shutdown_connections 2024-09-10T10:05:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.181+0000 7f3708475740 1 -- 172.21.15.45:0/328366598 wait complete. 2024-09-10T10:05:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.182+0000 7f3708475740 1 -- start start 2024-09-10T10:05:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.182+0000 7f3708475740 1 Processor -- start 2024-09-10T10:05:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.184+0000 7f3708475740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff62920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.184+0000 7f3708475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.184+0000 7f3708475740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.184+0000 7f3708475740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac1ff659f0 con 0x55ac1ff57cc0 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.184+0000 7f3708475740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ac1fe45290 con 0x55ac1ff62e80 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.184+0000 7f3708475740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55ac1ff65b70 con 0x55ac1ff602b0 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.184+0000 7f3708475740 10 client.0 _ll_register_callbacks cb 0x55ac1fe76140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.185+0000 7f36fe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.185+0000 7f36fe7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:50318/0 (socket says 172.21.15.45:50318) 2024-09-10T10:05:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.185+0000 7f36fe7fc640 1 -- 172.21.15.45:0/1173052255 learned_addr learned my addr 172.21.15.45:0/1173052255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.186+0000 7f36ff7fe640 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.186+0000 7f36fe7fc640 1 -- 172.21.15.45:0/1173052255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 msgr2=0x55ac1ff6eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.186+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.186+0000 7f36fe7fc640 1 -- 172.21.15.45:0/1173052255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 msgr2=0x55ac1ff62920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.186+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff62920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.186+0000 7f36fe7fc640 1 -- 172.21.15.45:0/1173052255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1fbcf210 con 0x55ac1ff62e80 2024-09-10T10:05:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.187+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5fbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36f002ae30 tx=0x7f36f0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc50838822bfae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.190+0000 7f36ff7fe640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f36e80a2e30 0x7f36e80a52f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005fd0 tx=0x7f36f8011920 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.190+0000 7f36e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:05:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.190+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1173052255 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0036040 con 0x55ac1ff62e80 2024-09-10T10:05:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.190+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac1fec9a20 con 0x55ac1ff62e80 2024-09-10T10:05:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.191+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1173052255 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f36f0032aa0 con 0x55ac1ff62e80 2024-09-10T10:05:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.191+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1173052255 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f36f0032ca0 con 0x55ac1ff62e80 2024-09-10T10:05:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.191+0000 7f3708475740 10 client.25022 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.192+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ac1fc9b620 con 0x55ac1ff62e80 2024-09-10T10:05:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.193+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1173052255 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ac1fec9a20 con 0x55ac1ff62e80 2024-09-10T10:05:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.194+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1173052255 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f36f00046c0 con 0x55ac1ff62e80 2024-09-10T10:05:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.194+0000 7f3708475740 10 client.25022 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.194+0000 7f3708475740 10 client.25022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.194+0000 7f3708475740 20 client.25022 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.194+0000 7f3708475740 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff734f0 0x55ac1ff75970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.194+0000 7f3708475740 4 client.25022 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:05:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.194+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1fec3d70 con 0x55ac1ff734f0 2024-09-10T10:05:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.196+0000 7f36feffd640 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff734f0 0x55ac1ff75970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.198+0000 7f36feffd640 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff734f0 0x55ac1ff75970 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513abe3b8a54c99e server_cookie=7ae411c23b057006 in_seq=0 out_seq=0 2024-09-10T10:05:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.198+0000 7f36e2ff5640 10 client.25022 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.207+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1173052255 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac1fec3d70 con 0x55ac1ff734f0 2024-09-10T10:05:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.207+0000 7f36e2ff5640 10 client.25022 handle_command_reply: tid=0 2024-09-10T10:05:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.207+0000 7f3708475740 1 client.25022 shutdown 2024-09-10T10:05:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.208+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff62e80 msgr2=0x55ac1ff5fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.208+0000 7f3708475740 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5fbc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36f002ae30 tx=0x7f36f0009b30 comp rx=0 tx=0).stop 2024-09-10T10:05:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.208+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 shutdown_connections 2024-09-10T10:05:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.208+0000 7f3708475740 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff734f0 0x55ac1ff75970 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.208+0000 7f3708475740 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.208+0000 7f3708475740 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5fbc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.208+0000 7f3708475740 1 --2- 172.21.15.45:0/1173052255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff62920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.209+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 >> 172.21.15.45:0/1173052255 conn(0x55ac1ff54230 msgr2=0x55ac1ff59770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.209+0000 7f36e2ff5640 0 client.25022 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.209+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 shutdown_connections 2024-09-10T10:05:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.209+0000 7f3708475740 1 -- 172.21.15.45:0/1173052255 wait complete. 2024-09-10T10:05:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.209+0000 7f3708475740 20 client.25022 trim_cache size 0 max 16384 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.289+0000 7f3708475740 1 Processor -- start 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.289+0000 7f3708475740 1 -- start start 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.289+0000 7f3708475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff62920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.289+0000 7f3708475740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.289+0000 7f3708475740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.290+0000 7f3708475740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec00c780 con 0x55ac1ff62e80 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.290+0000 7f3708475740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec00ef50 con 0x55ac1ff57cc0 2024-09-10T10:05:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.290+0000 7f3708475740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec00cd40 con 0x55ac1ff602b0 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.290+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.291+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52034/0 (socket says 172.21.15.45:52034) 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.291+0000 7f36ff7fe640 1 -- 172.21.15.45:0/1721526023 learned_addr learned my addr 172.21.15.45:0/1721526023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.292+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.292+0000 7f36feffd640 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff62920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.293+0000 7f36ff7fe640 1 -- 172.21.15.45:0/1721526023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 msgr2=0x55ac1ff6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.293+0000 7f36ff7fe640 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.293+0000 7f36ff7fe640 1 -- 172.21.15.45:0/1721526023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 msgr2=0x55ac1ff62920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.293+0000 7f36ff7fe640 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff62920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.293+0000 7f36ff7fe640 1 -- 172.21.15.45:0/1721526023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1ff6f9d0 con 0x55ac1ff62e80 2024-09-10T10:05:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.294+0000 7f36ff7fe640 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5f460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f36f8003e10 tx=0x7f36f801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3709d2cff083b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.304+0000 7f36e4ff9640 1 -- 172.21.15.45:0/1721526023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8004d20 con 0x55ac1ff62e80 2024-09-10T10:05:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.304+0000 7f36e4ff9640 1 -- 172.21.15.45:0/1721526023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f800f9c0 con 0x55ac1ff62e80 2024-09-10T10:05:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.305+0000 7f3708475740 1 -- 172.21.15.45:0/1721526023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 msgr2=0x55ac1ff5f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.305+0000 7f3708475740 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5f460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f36f8003e10 tx=0x7f36f801c000 comp rx=0 tx=0).stop 2024-09-10T10:05:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.308+0000 7f3708475740 1 -- 172.21.15.45:0/1721526023 shutdown_connections 2024-09-10T10:05:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.308+0000 7f3708475740 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff5f460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.308+0000 7f3708475740 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.308+0000 7f3708475740 1 --2- 172.21.15.45:0/1721526023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff62920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.308+0000 7f3708475740 1 -- 172.21.15.45:0/1721526023 >> 172.21.15.45:0/1721526023 conn(0x55ac1ff54230 msgr2=0x55ac1ff5e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.310+0000 7f3708475740 1 -- 172.21.15.45:0/1721526023 shutdown_connections 2024-09-10T10:05:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.310+0000 7f3708475740 1 -- 172.21.15.45:0/1721526023 wait complete. 2024-09-10T10:05:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.311+0000 7f3708475740 1 -- start start 2024-09-10T10:05:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.311+0000 7f3708475740 1 Processor -- start 2024-09-10T10:05:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.312+0000 7f3708475740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff7a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.312+0000 7f3708475740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.313+0000 7f36feffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff7a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.313+0000 7f36feffd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff7a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:50348/0 (socket says 172.21.15.45:50348) 2024-09-10T10:05:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.313+0000 7f36feffd640 1 -- 172.21.15.45:0/1594328904 learned_addr learned my addr 172.21.15.45:0/1594328904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:05:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.313+0000 7f3708475740 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff7acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.314+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.314+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec00d410 con 0x55ac1ff62e80 2024-09-10T10:05:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.314+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec00e880 con 0x55ac1ff57cc0 2024-09-10T10:05:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.314+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec00dae0 con 0x55ac1ff602b0 2024-09-10T10:05:58.316 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.315+0000 7f3708475740 10 client.0 _ll_register_callbacks cb 0x7f36f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:05:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.319+0000 7f36fe7fc640 1 -- 172.21.15.45:0/1594328904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 msgr2=0x55ac1ff7a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.319+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff7a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.319+0000 7f36fe7fc640 1 -- 172.21.15.45:0/1594328904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 msgr2=0x55ac1ff7acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:05:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.319+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff7acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.319+0000 7f36fe7fc640 1 -- 172.21.15.45:0/1594328904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1ff69eb0 con 0x55ac1ff602b0 2024-09-10T10:05:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.321+0000 7f36fe7fc640 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff7a7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36f0004ae0 tx=0x7f36f00048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a129932edc7baea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.321+0000 7f36e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:05:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.323+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1594328904 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0036040 con 0x55ac1ff602b0 2024-09-10T10:05:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.324+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f800f9c0 con 0x55ac1ff602b0 2024-09-10T10:05:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.324+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1594328904 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f36f0004030 con 0x55ac1ff602b0 2024-09-10T10:05:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.324+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1594328904 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f36f002e020 con 0x55ac1ff602b0 2024-09-10T10:05:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.324+0000 7f3708475740 10 client.24949 fetch_fsmap learned FSMap version 19 2024-09-10T10:05:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.324+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ac1ff6b120 con 0x55ac1ff602b0 2024-09-10T10:05:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.325+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1594328904 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36f800f9c0 con 0x55ac1ff602b0 2024-09-10T10:05:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.326+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1594328904 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f36f00035f0 con 0x55ac1ff602b0 2024-09-10T10:05:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.327+0000 7f3708475740 10 client.24949 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:05:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.327+0000 7f3708475740 10 client.24949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:05:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.327+0000 7f3708475740 20 client.24949 populate_metadata read hostname 'smithi045' 2024-09-10T10:05:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.327+0000 7f3708475740 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff6b810 0x55ac1ffe0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.327+0000 7f3708475740 4 client.24949 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:05:58.329 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.327+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac1fe40900 con 0x55ac1ff6b810 2024-09-10T10:05:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.329+0000 7f36feffd640 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff6b810 0x55ac1ffe0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.343+0000 7f36e2ff5640 10 client.24949 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:05:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:05:58.343+0000 7f36feffd640 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff6b810 0x55ac1ffe0260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c54c41e9103a5a server_cookie=57f1d152aa055bcd in_seq=0 out_seq=0 2024-09-10T10:05:59.406 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:05:59 smithi045 ceph-mon[31114]: pgmap v86: 97 pgs: 97 active+clean; 20 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 834 MiB/s wr, 1.28k op/s 2024-09-10T10:05:59.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:05:59 smithi047 ceph-mon[35244]: pgmap v86: 97 pgs: 97 active+clean; 20 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 834 MiB/s wr, 1.28k op/s 2024-09-10T10:05:59.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:05:59 smithi162 ceph-mon[34963]: pgmap v86: 97 pgs: 97 active+clean; 20 GiB data, 105 GiB used, 968 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 834 MiB/s wr, 1.28k op/s 2024-09-10T10:06:01.451 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:01 smithi162 ceph-mon[34963]: pgmap v87: 97 pgs: 97 active+clean; 20 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 992 MiB/s wr, 1.55k op/s 2024-09-10T10:06:01.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:01 smithi045 ceph-mon[31114]: pgmap v87: 97 pgs: 97 active+clean; 20 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 992 MiB/s wr, 1.55k op/s 2024-09-10T10:06:01.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:01 smithi047 ceph-mon[35244]: pgmap v87: 97 pgs: 97 active+clean; 20 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 992 MiB/s wr, 1.55k op/s 2024-09-10T10:06:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f36e2ff5640 1 -- 172.21.15.45:0/1594328904 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac1fe40900 con 0x55ac1ff6b810 2024-09-10T10:06:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f36e2ff5640 10 client.24949 handle_command_reply: tid=0 2024-09-10T10:06:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 client.24949 shutdown 2024-09-10T10:06:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 msgr2=0x55ac1ff7a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff7a7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36f0004ae0 tx=0x7f36f00048f0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 shutdown_connections 2024-09-10T10:06:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55ac1ff6b810 0x55ac1ffe0260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac1ff62e80 0x55ac1ff7acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55ac1ff602b0 0x55ac1ff7a7b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f36e2ff5640 0 client.24949 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 --2- 172.21.15.45:0/1594328904 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac1ff57cc0 0x55ac1ff7a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.670+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 >> 172.21.15.45:0/1594328904 conn(0x55ac1ff54230 msgr2=0x55ac1ff5e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.671+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 shutdown_connections 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.671+0000 7f3708475740 1 -- 172.21.15.45:0/1594328904 wait complete. 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.671+0000 7f3708475740 20 client.24949 trim_cache size 0 max 16384 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:06:02.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:06:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:06:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f36e80a2e30 msgr2=0x7f36e80a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f36e80a2e30 0x7f36e80a52f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36f8005fd0 tx=0x7f36f8011920 comp rx=0 tx=0).stop 2024-09-10T10:06:02.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3700074620 msgr2=0x7f37000847c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3700074620 0x7f37000847c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f36f400e9f0 tx=0x7f36f400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 shutdown_connections 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f370007f8d0 0x7f370007fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f36e80a2e30 0x7f36e80a52f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3700109460 0x7f3700084d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 --2- 172.21.15.45:0/3581970433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3700074620 0x7f37000847c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 >> 172.21.15.45:0/3581970433 conn(0x7f370006cbf0 msgr2=0x7f3700072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 shutdown_connections 2024-09-10T10:06:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:02.672+0000 7f36fffff640 1 -- 172.21.15.45:0/3581970433 wait complete. 2024-09-10T10:06:03.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:03 smithi162 ceph-mon[34963]: pgmap v88: 97 pgs: 97 active+clean; 20 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 741 MiB/s wr, 1.13k op/s 2024-09-10T10:06:03.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:03 smithi045 ceph-mon[31114]: pgmap v88: 97 pgs: 97 active+clean; 20 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 741 MiB/s wr, 1.13k op/s 2024-09-10T10:06:03.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:03 smithi047 ceph-mon[35244]: pgmap v88: 97 pgs: 97 active+clean; 20 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 741 MiB/s wr, 1.13k op/s 2024-09-10T10:06:05.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:05 smithi162 ceph-mon[34963]: pgmap v89: 97 pgs: 97 active+clean; 20 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 GiB/s wr, 1.95k op/s 2024-09-10T10:06:05.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:05 smithi045 ceph-mon[31114]: pgmap v89: 97 pgs: 97 active+clean; 20 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 GiB/s wr, 1.95k op/s 2024-09-10T10:06:05.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:05 smithi047 ceph-mon[35244]: pgmap v89: 97 pgs: 97 active+clean; 20 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 GiB/s wr, 1.95k op/s 2024-09-10T10:06:07.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:07 smithi162 ceph-mon[34963]: pgmap v90: 97 pgs: 97 active+clean; 20 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 754 MiB/s wr, 1.55k op/s 2024-09-10T10:06:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:07 smithi045 ceph-mon[31114]: pgmap v90: 97 pgs: 97 active+clean; 20 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 754 MiB/s wr, 1.55k op/s 2024-09-10T10:06:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:07 smithi047 ceph-mon[35244]: pgmap v90: 97 pgs: 97 active+clean; 20 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 754 MiB/s wr, 1.55k op/s 2024-09-10T10:06:09.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:09 smithi162 ceph-mon[34963]: pgmap v91: 97 pgs: 97 active+clean; 20 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 792 MiB/s wr, 1.78k op/s 2024-09-10T10:06:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:09 smithi045 ceph-mon[31114]: pgmap v91: 97 pgs: 97 active+clean; 20 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 792 MiB/s wr, 1.78k op/s 2024-09-10T10:06:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:09 smithi047 ceph-mon[35244]: pgmap v91: 97 pgs: 97 active+clean; 20 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 792 MiB/s wr, 1.78k op/s 2024-09-10T10:06:11.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:11 smithi045 ceph-mon[31114]: pgmap v92: 97 pgs: 97 active+clean; 22 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 939 MiB/s wr, 2.63k op/s 2024-09-10T10:06:11.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:11 smithi047 ceph-mon[35244]: pgmap v92: 97 pgs: 97 active+clean; 22 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 939 MiB/s wr, 2.63k op/s 2024-09-10T10:06:11.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:11 smithi162 ceph-mon[34963]: pgmap v92: 97 pgs: 97 active+clean; 22 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 939 MiB/s wr, 2.63k op/s 2024-09-10T10:06:13.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:13 smithi162 ceph-mon[34963]: pgmap v93: 97 pgs: 97 active+clean; 22 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 684 MiB/s wr, 2.18k op/s 2024-09-10T10:06:13.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:13 smithi045 ceph-mon[31114]: pgmap v93: 97 pgs: 97 active+clean; 22 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 684 MiB/s wr, 2.18k op/s 2024-09-10T10:06:13.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:13 smithi047 ceph-mon[35244]: pgmap v93: 97 pgs: 97 active+clean; 22 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 684 MiB/s wr, 2.18k op/s 2024-09-10T10:06:15.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:15 smithi162 ceph-mon[34963]: pgmap v94: 97 pgs: 97 active+clean; 23 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 994 MiB/s wr, 4.43k op/s 2024-09-10T10:06:15.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:15 smithi045 ceph-mon[31114]: pgmap v94: 97 pgs: 97 active+clean; 23 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 994 MiB/s wr, 4.43k op/s 2024-09-10T10:06:15.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:15 smithi047 ceph-mon[35244]: pgmap v94: 97 pgs: 97 active+clean; 23 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 994 MiB/s wr, 4.43k op/s 2024-09-10T10:06:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.931+0000 7f085e265640 1 -- 172.21.15.45:0/50916191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08581095d0 msgr2=0x7f0858109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.931+0000 7f085e265640 1 --2- 172.21.15.45:0/50916191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08581095d0 0x7f0858109a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0850009c10 tx=0x7f085002a720 comp rx=0 tx=0).stop 2024-09-10T10:06:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.937+0000 7f085e265640 1 -- 172.21.15.45:0/50916191 shutdown_connections 2024-09-10T10:06:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.937+0000 7f085e265640 1 --2- 172.21.15.45:0/50916191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085811b140 0x7f085811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.937+0000 7f085e265640 1 --2- 172.21.15.45:0/50916191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08581095d0 0x7f0858109a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.937+0000 7f085e265640 1 --2- 172.21.15.45:0/50916191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581091f0 0x7f0858110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.937+0000 7f085e265640 1 -- 172.21.15.45:0/50916191 >> 172.21.15.45:0/50916191 conn(0x7f085806ce00 msgr2=0x7f085806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.937+0000 7f085e265640 1 -- 172.21.15.45:0/50916191 shutdown_connections 2024-09-10T10:06:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.937+0000 7f085e265640 1 -- 172.21.15.45:0/50916191 wait complete. 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.938+0000 7f085e265640 1 Processor -- start 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.938+0000 7f085e265640 1 -- start start 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.939+0000 7f085e265640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08581091f0 0x7f085811a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.939+0000 7f085e265640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581095d0 0x7f0858116850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.939+0000 7f085e265640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085811b140 0x7f0858116d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.939+0000 7f085e265640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0858113360 con 0x7f08581095d0 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.939+0000 7f085e265640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08581131e0 con 0x7f08581091f0 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.939+0000 7f085e265640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f08581134e0 con 0x7f085811b140 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.940+0000 7f08577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581095d0 0x7f0858116850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.940+0000 7f08577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581095d0 0x7f0858116850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34288/0 (socket says 172.21.15.45:34288) 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.940+0000 7f08577fe640 1 -- 172.21.15.45:0/798578135 learned_addr learned my addr 172.21.15.45:0/798578135 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.940+0000 7f085c802640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085811b140 0x7f0858116d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.941+0000 7f08577fe640 1 -- 172.21.15.45:0/798578135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085811b140 msgr2=0x7f0858116d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.941+0000 7f08577fe640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085811b140 0x7f0858116d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.941+0000 7f08577fe640 1 -- 172.21.15.45:0/798578135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08581091f0 msgr2=0x7f085811a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.941+0000 7f08577fe640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08581091f0 0x7f085811a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.941+0000 7f08577fe640 1 -- 172.21.15.45:0/798578135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0858117350 con 0x7f08581095d0 2024-09-10T10:06:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.941+0000 7f08577fe640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581095d0 0x7f0858116850 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0850031a90 tx=0x7f0850030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.942+0000 7f08557fa640 1 -- 172.21.15.45:0/798578135 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850041020 con 0x7f08581095d0 2024-09-10T10:06:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.942+0000 7f08557fa640 1 -- 172.21.15.45:0/798578135 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085003cc10 con 0x7f08581095d0 2024-09-10T10:06:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.942+0000 7f08557fa640 1 -- 172.21.15.45:0/798578135 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850036040 con 0x7f08581095d0 2024-09-10T10:06:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.942+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08581175e0 con 0x7f08581095d0 2024-09-10T10:06:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.942+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085807c6b0 con 0x7f08581095d0 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.948+0000 7f0861007740 1 Processor -- start 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f0861007740 1 -- start start 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f0861007740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eafaf0 0x5637b5eafed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f0861007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f0861007740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5ebd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f0861007740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5637b5e142f0 con 0x5637b5ebacb0 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f0861007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5637b5dbc270 con 0x5637b5eb0410 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f0861007740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5637b5e102b0 con 0x5637b5eafaf0 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.949+0000 7f08557fa640 1 -- 172.21.15.45:0/798578135 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0850044020 con 0x7f08581095d0 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.950+0000 7f0857fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.950+0000 7f0857fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eba770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:49100/0 (socket says 172.21.15.45:49100) 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.950+0000 7f0857fff640 1 -- 172.21.15.45:0/1814687000 learned_addr learned my addr 172.21.15.45:0/1814687000 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.950+0000 7f085c802640 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eafaf0 0x5637b5eafed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.951+0000 7f0857fff640 1 -- 172.21.15.45:0/1814687000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eafaf0 msgr2=0x5637b5eafed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.951+0000 7f0857fff640 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eafaf0 0x5637b5eafed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.951+0000 7f0857fff640 1 -- 172.21.15.45:0/1814687000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 msgr2=0x5637b5ebd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.951+0000 7f0857fff640 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5ebd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.951+0000 7f0857fff640 1 -- 172.21.15.45:0/1814687000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b5dbb340 con 0x5637b5eb0410 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.951+0000 7f08557fa640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f08400a3000 0x7f08400a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.951+0000 7f08557fa640 1 -- 172.21.15.45:0/798578135 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f08500f0640 con 0x7f08581095d0 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.952+0000 7f085c802640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f08400a3000 0x7f08400a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.952+0000 7f0857fff640 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eba770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f084801ab40 tx=0x7f084801c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9538d4f7b76adc4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.953+0000 7f085c802640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f08400a3000 0x7f08400a54c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f084c006fd0 tx=0x7f084c008040 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.954+0000 7f083e7fc640 1 -- 172.21.15.45:0/1814687000 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848013030 con 0x5637b5eb0410 2024-09-10T10:06:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.955+0000 7f083e7fc640 1 -- 172.21.15.45:0/1814687000 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0848005020 con 0x5637b5eb0410 2024-09-10T10:06:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.955+0000 7f083e7fc640 1 -- 172.21.15.45:0/1814687000 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848004d60 con 0x5637b5eb0410 2024-09-10T10:06:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.958+0000 7f0861007740 1 -- 172.21.15.45:0/1814687000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 msgr2=0x5637b5eba770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.958+0000 7f0861007740 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eba770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f084801ab40 tx=0x7f084801c2d0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.958+0000 7f0861007740 1 -- 172.21.15.45:0/1814687000 shutdown_connections 2024-09-10T10:06:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.958+0000 7f0861007740 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5ebd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.958+0000 7f0861007740 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eba770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.958+0000 7f0861007740 1 --2- 172.21.15.45:0/1814687000 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eafaf0 0x5637b5eafed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.958+0000 7f0861007740 1 -- 172.21.15.45:0/1814687000 >> 172.21.15.45:0/1814687000 conn(0x5637b5eac060 msgr2=0x5637b5eac440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.959+0000 7f0861007740 1 -- 172.21.15.45:0/1814687000 shutdown_connections 2024-09-10T10:06:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.959+0000 7f0861007740 1 -- 172.21.15.45:0/1814687000 wait complete. 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.959+0000 7f0861007740 1 -- start start 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 1 Processor -- start 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eafaf0 0x5637b5eb7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eb81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5eb86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5637b5ebd880 con 0x5637b5eafaf0 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5637b5d9d290 con 0x5637b5eb0410 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5637b5ebda00 con 0x5637b5ebacb0 2024-09-10T10:06:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.960+0000 7f0861007740 10 client.0 _ll_register_callbacks cb 0x5637b5dce140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eafaf0 0x5637b5eb7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eafaf0 0x5637b5eb7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34312/0 (socket says 172.21.15.45:34312) 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 -- 172.21.15.45:0/1636565162 learned_addr learned my addr 172.21.15.45:0/1636565162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f08577fe640 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eb81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f085c802640 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5eb86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 -- 172.21.15.45:0/1636565162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 msgr2=0x5637b5eb86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5eb86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 -- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 msgr2=0x5637b5eb81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eb81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.961+0000 7f0857fff640 1 -- 172.21.15.45:0/1636565162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b5b27210 con 0x5637b5eafaf0 2024-09-10T10:06:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.962+0000 7f0857fff640 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eafaf0 0x5637b5eb7c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0848004f10 tx=0x7f0848004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a55fbca1cb3c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.962+0000 7f08377fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.962+0000 7f08377fe640 1 -- 172.21.15.45:0/1636565162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08480055e0 con 0x5637b5eafaf0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.962+0000 7f08377fe640 1 -- 172.21.15.45:0/1636565162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f0848043480 con 0x5637b5eafaf0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.963+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0848005020 con 0x5637b5eafaf0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.963+0000 7f08377fe640 1 -- 172.21.15.45:0/1636565162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0848043680 con 0x5637b5eafaf0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.963+0000 7f0861007740 10 client.15330 fetch_fsmap learned FSMap version 19 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.963+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5637b5bf3620 con 0x5637b5eafaf0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f08377fe640 1 -- 172.21.15.45:0/1636565162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0848048070 con 0x5637b5eafaf0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f08377fe640 1 -- 172.21.15.45:0/1636565162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f0848040070 con 0x5637b5eafaf0 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f0861007740 10 client.15330 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f0861007740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f0861007740 20 client.15330 populate_metadata read hostname 'smithi045' 2024-09-10T10:06:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f0861007740 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5ecabb0 0x5637b5eccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f0861007740 4 client.15330 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:06:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.964+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0848005020 con 0x5637b5ecabb0 2024-09-10T10:06:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.965+0000 7f08577fe640 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5ecabb0 0x5637b5eccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.967+0000 7f08377fe640 10 client.15330 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:15.970 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.967+0000 7f08577fe640 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5ecabb0 0x5637b5eccfa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69628c43e4770d8 server_cookie=cad61c82327e2420 in_seq=0 out_seq=0 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.970+0000 7f08377fe640 1 -- 172.21.15.45:0/1636565162 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0848005020 con 0x5637b5ecabb0 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.970+0000 7f08377fe640 10 client.15330 handle_command_reply: tid=0 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.970+0000 7f0861007740 1 client.15330 shutdown 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eafaf0 msgr2=0x5637b5eb7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eafaf0 0x5637b5eb7c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0848004f10 tx=0x7f0848004f40 comp rx=0 tx=0).stop 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 shutdown_connections 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5ecabb0 0x5637b5eccfa0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f08377fe640 0 client.15330 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5eb86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eb0410 0x5637b5eb81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 --2- 172.21.15.45:0/1636565162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eafaf0 0x5637b5eb7c70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 >> 172.21.15.45:0/1636565162 conn(0x5637b5eac060 msgr2=0x5637b5eb1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 shutdown_connections 2024-09-10T10:06:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 1 -- 172.21.15.45:0/1636565162 wait complete. 2024-09-10T10:06:15.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:15.971+0000 7f0861007740 20 client.15330 trim_cache size 0 max 16384 2024-09-10T10:06:16.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.008+0000 7f0861007740 1 Processor -- start 2024-09-10T10:06:16.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.008+0000 7f0861007740 1 -- start start 2024-09-10T10:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.008+0000 7f0861007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5eb8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.009+0000 7f0861007740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eb0410 0x5637b5ed27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.009+0000 7f0861007740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5ed2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.009+0000 7f0861007740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0858117d30 con 0x5637b5eb0410 2024-09-10T10:06:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.009+0000 7f0861007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08581c5630 con 0x5637b5eafaf0 2024-09-10T10:06:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.009+0000 7f0861007740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0858083840 con 0x5637b5ebacb0 2024-09-10T10:06:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.010+0000 7f08577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eb0410 0x5637b5ed27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.010+0000 7f08577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eb0410 0x5637b5ed27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34326/0 (socket says 172.21.15.45:34326) 2024-09-10T10:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.010+0000 7f08577fe640 1 -- 172.21.15.45:0/1440901291 learned_addr learned my addr 172.21.15.45:0/1440901291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.010+0000 7f0857fff640 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5eb8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.010+0000 7f085c802640 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5ed2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.011+0000 7f08577fe640 1 -- 172.21.15.45:0/1440901291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 msgr2=0x5637b5ed2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.011+0000 7f08577fe640 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5ed2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.011+0000 7f08577fe640 1 -- 172.21.15.45:0/1440901291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 msgr2=0x5637b5eb8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.011+0000 7f08577fe640 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5eb8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.011+0000 7f08577fe640 1 -- 172.21.15.45:0/1440901291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637b5eaf870 con 0x5637b5eb0410 2024-09-10T10:06:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.011+0000 7f08577fe640 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eb0410 0x5637b5ed27a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08500b9500 tx=0x7f0850009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9908c3f6b8d272eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.012+0000 7f083dffb640 1 -- 172.21.15.45:0/1440901291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500da070 con 0x5637b5eb0410 2024-09-10T10:06:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.012+0000 7f083dffb640 1 -- 172.21.15.45:0/1440901291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0848005020 con 0x5637b5eb0410 2024-09-10T10:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 -- 172.21.15.45:0/1440901291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eb0410 msgr2=0x5637b5ed27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eb0410 0x5637b5ed27a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08500b9500 tx=0x7f0850009500 comp rx=0 tx=0).stop 2024-09-10T10:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 -- 172.21.15.45:0/1440901291 shutdown_connections 2024-09-10T10:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5ebacb0 0x5637b5ed2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5eb0410 0x5637b5ed27a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 --2- 172.21.15.45:0/1440901291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5eb8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 -- 172.21.15.45:0/1440901291 >> 172.21.15.45:0/1440901291 conn(0x5637b5eac060 msgr2=0x5637b5eb9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 -- 172.21.15.45:0/1440901291 shutdown_connections 2024-09-10T10:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.014+0000 7f0861007740 1 -- 172.21.15.45:0/1440901291 wait complete. 2024-09-10T10:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.015+0000 7f0861007740 1 -- start start 2024-09-10T10:06:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 1 Processor -- start 2024-09-10T10:06:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5ec31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5eb4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eb4c80 0x5637b5eb5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0858083f10 con 0x5637b5ebacb0 2024-09-10T10:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08581c5030 con 0x5637b5eafaf0 2024-09-10T10:06:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f08580845e0 con 0x5637b5eb4c80 2024-09-10T10:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.016+0000 7f0861007740 10 client.0 _ll_register_callbacks cb 0x7f084801aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.017+0000 7f08577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5eb4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.017+0000 7f08577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5eb4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34340/0 (socket says 172.21.15.45:34340) 2024-09-10T10:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.017+0000 7f08577fe640 1 -- 172.21.15.45:0/1414804796 learned_addr learned my addr 172.21.15.45:0/1414804796 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.018+0000 7f0857fff640 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5ec31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.018+0000 7f08577fe640 1 -- 172.21.15.45:0/1414804796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eb4c80 msgr2=0x5637b5eb5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.018+0000 7f08577fe640 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eb4c80 0x5637b5eb5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.018+0000 7f08577fe640 1 -- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 msgr2=0x5637b5ec31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.018+0000 7f08577fe640 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5ec31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.018+0000 7f08577fe640 1 -- 172.21.15.45:0/1414804796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637b5f34770 con 0x5637b5ebacb0 2024-09-10T10:06:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.019+0000 7f08577fe640 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5eb4740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08500c5270 tx=0x7f0850005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651d898b57ed9324 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.019+0000 7f08377fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:06:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.019+0000 7f08377fe640 1 -- 172.21.15.45:0/1414804796 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08500da050 con 0x5637b5ebacb0 2024-09-10T10:06:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.019+0000 7f08377fe640 1 -- 172.21.15.45:0/1414804796 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f08500c5310 con 0x5637b5ebacb0 2024-09-10T10:06:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.019+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5637b5e21a20 con 0x5637b5ebacb0 2024-09-10T10:06:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.020+0000 7f08377fe640 1 -- 172.21.15.45:0/1414804796 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f08500bde60 con 0x5637b5ebacb0 2024-09-10T10:06:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.021+0000 7f0861007740 10 client.15336 fetch_fsmap learned FSMap version 19 2024-09-10T10:06:16.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.021+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5637b5f359e0 con 0x5637b5ebacb0 2024-09-10T10:06:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f08377fe640 1 -- 172.21.15.45:0/1414804796 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5637b5e21a20 con 0x5637b5ebacb0 2024-09-10T10:06:16.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f08377fe640 1 -- 172.21.15.45:0/1414804796 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f08500db290 con 0x5637b5ebacb0 2024-09-10T10:06:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f0861007740 10 client.15336 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:06:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f0861007740 10 client.15336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:06:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f0861007740 20 client.15336 populate_metadata read hostname 'smithi045' 2024-09-10T10:06:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f0861007740 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5f36060 0x5637b5f38480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f0861007740 4 client.15336 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:06:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.022+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637b5e1bd70 con 0x5637b5f36060 2024-09-10T10:06:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.024+0000 7f0857fff640 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5f36060 0x5637b5f38480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.027+0000 7f08377fe640 10 client.15336 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:16.027+0000 7f0857fff640 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5f36060 0x5637b5f38480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a1db76481884a7 server_cookie=686d87d6bc2dbadd in_seq=0 out_seq=0 2024-09-10T10:06:17.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:17 smithi162 ceph-mon[34963]: pgmap v95: 97 pgs: 97 active+clean; 23 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 957 KiB/s rd, 734 MiB/s wr, 3.90k op/s 2024-09-10T10:06:17.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:17 smithi045 ceph-mon[31114]: pgmap v95: 97 pgs: 97 active+clean; 23 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 957 KiB/s rd, 734 MiB/s wr, 3.90k op/s 2024-09-10T10:06:17.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:17 smithi047 ceph-mon[35244]: pgmap v95: 97 pgs: 97 active+clean; 23 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 957 KiB/s rd, 734 MiB/s wr, 3.90k op/s 2024-09-10T10:06:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.616+0000 7f08377fe640 1 -- 172.21.15.45:0/1414804796 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637b5e1bd70 con 0x5637b5f36060 2024-09-10T10:06:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.616+0000 7f08377fe640 10 client.15336 handle_command_reply: tid=0 2024-09-10T10:06:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.616+0000 7f0861007740 1 client.15336 shutdown 2024-09-10T10:06:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 msgr2=0x5637b5eb4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5eb4740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08500c5270 tx=0x7f0850005e30 comp rx=0 tx=0).stop 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 shutdown_connections 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5637b5f36060 0x5637b5f38480 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5637b5eb4c80 0x5637b5eb5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637b5ebacb0 0x5637b5eb4740 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 --2- 172.21.15.45:0/1414804796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5637b5eafaf0 0x5637b5ec31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 >> 172.21.15.45:0/1414804796 conn(0x5637b5eac060 msgr2=0x5637b5eaa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f08377fe640 0 client.15336 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.617+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 shutdown_connections 2024-09-10T10:06:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.618+0000 7f0861007740 1 -- 172.21.15.45:0/1414804796 wait complete. 2024-09-10T10:06:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.618+0000 7f0861007740 20 client.15336 trim_cache size 0 max 16384 2024-09-10T10:06:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:06:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:06:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:06:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:06:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.621+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f08400a3000 msgr2=0x7f08400a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.622+0000 7f085e265640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f08400a3000 0x7f08400a54c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f084c006fd0 tx=0x7f084c008040 comp rx=0 tx=0).stop 2024-09-10T10:06:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.622+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581095d0 msgr2=0x7f0858116850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.622+0000 7f085e265640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581095d0 0x7f0858116850 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0850031a90 tx=0x7f0850030890 comp rx=0 tx=0).stop 2024-09-10T10:06:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.622+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 shutdown_connections 2024-09-10T10:06:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.622+0000 7f085e265640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f085811b140 0x7f0858116d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.622+0000 7f085e265640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08581095d0 0x7f0858116850 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.622+0000 7f085e265640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f08400a3000 0x7f08400a54c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.623+0000 7f085e265640 1 --2- 172.21.15.45:0/798578135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08581091f0 0x7f085811a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.623+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 >> 172.21.15.45:0/798578135 conn(0x7f085806ce00 msgr2=0x7f0858071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.623+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 shutdown_connections 2024-09-10T10:06:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:19.623+0000 7f085e265640 1 -- 172.21.15.45:0/798578135 wait complete. 2024-09-10T10:06:19.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:19 smithi162 ceph-mon[34963]: pgmap v96: 97 pgs: 97 active+clean; 24 GiB data, 129 GiB used, 944 GiB / 1.0 TiB avail; 957 KiB/s rd, 800 MiB/s wr, 4.33k op/s 2024-09-10T10:06:19.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:19 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:06:19.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:19 smithi045 ceph-mon[31114]: pgmap v96: 97 pgs: 97 active+clean; 24 GiB data, 129 GiB used, 944 GiB / 1.0 TiB avail; 957 KiB/s rd, 800 MiB/s wr, 4.33k op/s 2024-09-10T10:06:19.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:19 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:06:19.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:19 smithi047 ceph-mon[35244]: pgmap v96: 97 pgs: 97 active+clean; 24 GiB data, 129 GiB used, 944 GiB / 1.0 TiB avail; 957 KiB/s rd, 800 MiB/s wr, 4.33k op/s 2024-09-10T10:06:19.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:19 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:06:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:21 smithi162 ceph-mon[34963]: pgmap v97: 97 pgs: 97 active+clean; 26 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 MiB/s wr, 4.97k op/s 2024-09-10T10:06:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:21 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:06:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:21 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:06:21.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:21 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:06:21.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:21 smithi045 ceph-mon[31114]: pgmap v97: 97 pgs: 97 active+clean; 26 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 MiB/s wr, 4.97k op/s 2024-09-10T10:06:21.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:21 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:06:21.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:21 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:06:21.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:21 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:06:21.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:21 smithi047 ceph-mon[35244]: pgmap v97: 97 pgs: 97 active+clean; 26 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 MiB/s wr, 4.97k op/s 2024-09-10T10:06:21.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:21 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:06:21.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:21 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:06:21.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:21 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:06:23.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:23 smithi162 ceph-mon[34963]: pgmap v98: 97 pgs: 97 active+clean; 26 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 691 MiB/s wr, 3.96k op/s 2024-09-10T10:06:23.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:23 smithi045 ceph-mon[31114]: pgmap v98: 97 pgs: 97 active+clean; 26 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 691 MiB/s wr, 3.96k op/s 2024-09-10T10:06:23.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:23 smithi047 ceph-mon[35244]: pgmap v98: 97 pgs: 97 active+clean; 26 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 691 MiB/s wr, 3.96k op/s 2024-09-10T10:06:26.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:25 smithi162 ceph-mon[34963]: pgmap v99: 97 pgs: 97 active+clean; 28 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 1006 MiB/s wr, 4.57k op/s 2024-09-10T10:06:26.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:25 smithi045 ceph-mon[31114]: pgmap v99: 97 pgs: 97 active+clean; 28 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 1006 MiB/s wr, 4.57k op/s 2024-09-10T10:06:26.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:25 smithi047 ceph-mon[35244]: pgmap v99: 97 pgs: 97 active+clean; 28 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 1006 MiB/s wr, 4.57k op/s 2024-09-10T10:06:27.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:26 smithi162 ceph-mon[34963]: pgmap v100: 97 pgs: 97 active+clean; 28 GiB data, 141 GiB used, 932 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 744 MiB/s wr, 2.42k op/s 2024-09-10T10:06:27.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:26 smithi045 ceph-mon[31114]: pgmap v100: 97 pgs: 97 active+clean; 28 GiB data, 141 GiB used, 932 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 744 MiB/s wr, 2.42k op/s 2024-09-10T10:06:27.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:26 smithi047 ceph-mon[35244]: pgmap v100: 97 pgs: 97 active+clean; 28 GiB data, 141 GiB used, 932 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 744 MiB/s wr, 2.42k op/s 2024-09-10T10:06:29.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:28 smithi047 ceph-mon[35244]: pgmap v101: 97 pgs: 97 active+clean; 29 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 816 MiB/s wr, 2.56k op/s 2024-09-10T10:06:29.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:28 smithi162 ceph-mon[34963]: pgmap v101: 97 pgs: 97 active+clean; 29 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 816 MiB/s wr, 2.56k op/s 2024-09-10T10:06:29.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:28 smithi045 ceph-mon[31114]: pgmap v101: 97 pgs: 97 active+clean; 29 GiB data, 143 GiB used, 930 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 816 MiB/s wr, 2.56k op/s 2024-09-10T10:06:31.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:31 smithi162 ceph-mon[34963]: pgmap v102: 97 pgs: 97 active+clean; 30 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 959 MiB/s wr, 3.04k op/s 2024-09-10T10:06:31.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:31 smithi045 ceph-mon[31114]: pgmap v102: 97 pgs: 97 active+clean; 30 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 959 MiB/s wr, 3.04k op/s 2024-09-10T10:06:31.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:31 smithi047 ceph-mon[35244]: pgmap v102: 97 pgs: 97 active+clean; 30 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 959 MiB/s wr, 3.04k op/s 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 -- 172.21.15.45:0/48705344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f400ffdd0 msgr2=0x7f6f401001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/48705344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f400ffdd0 0x7f6f401001b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009900 tx=0x7f6f3002a440 comp rx=0 tx=0).stop 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 -- 172.21.15.45:0/48705344 shutdown_connections 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/48705344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f4010a2a0 0x7f6f40110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/48705344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f40100780 0x7f6f40109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/48705344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f400ffdd0 0x7f6f401001b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 -- 172.21.15.45:0/48705344 >> 172.21.15.45:0/48705344 conn(0x7f6f400fba30 msgr2=0x7f6f400fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 -- 172.21.15.45:0/48705344 shutdown_connections 2024-09-10T10:06:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.858+0000 7f6f44bc4640 1 -- 172.21.15.45:0/48705344 wait complete. 2024-09-10T10:06:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.859+0000 7f6f44bc4640 1 Processor -- start 2024-09-10T10:06:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.859+0000 7f6f44bc4640 1 -- start start 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f44bc4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f400ffdd0 0x7f6f4019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f44bc4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4010a2a0 0x7f6f4019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f44bc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f4019ee00 0x7f6f4019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f44bc4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6f40114230 con 0x7f6f4019ee00 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f44bc4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401140b0 con 0x7f6f4010a2a0 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f44bc4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401143b0 con 0x7f6f400ffdd0 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f4019ee00 0x7f6f4019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f4019ee00 0x7f6f4019f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39090/0 (socket says 172.21.15.45:39090) 2024-09-10T10:06:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 -- 172.21.15.45:0/2668070187 learned_addr learned my addr 172.21.15.45:0/2668070187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 -- 172.21.15.45:0/2668070187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f400ffdd0 msgr2=0x7f6f4019a540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:06:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f400ffdd0 0x7f6f4019a540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 -- 172.21.15.45:0/2668070187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4010a2a0 msgr2=0x7f6f4019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4010a2a0 0x7f6f4019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.860+0000 7f6f3ed9d640 1 -- 172.21.15.45:0/2668070187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4019f8f0 con 0x7f6f4019ee00 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.861+0000 7f6f3ed9d640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f4019ee00 0x7f6f4019f2b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800b990 tx=0x7f6f3800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.861+0000 7f6f2f7fe640 1 -- 172.21.15.45:0/2668070187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38019070 con 0x7f6f4019ee00 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.861+0000 7f6f2f7fe640 1 -- 172.21.15.45:0/2668070187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3800cba0 con 0x7f6f4019ee00 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.861+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4019fbe0 con 0x7f6f4019ee00 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.861+0000 7f6f2f7fe640 1 -- 172.21.15.45:0/2668070187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38007650 con 0x7f6f4019ee00 2024-09-10T10:06:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.861+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f401a73d0 con 0x7f6f4019ee00 2024-09-10T10:06:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f2f7fe640 1 -- 172.21.15.45:0/2668070187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6f3801c070 con 0x7f6f4019ee00 2024-09-10T10:06:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 Processor -- start 2024-09-10T10:06:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 -- start start 2024-09-10T10:06:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bac0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bacb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bace0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556d3ba252f0 con 0x556d3bac1410 2024-09-10T10:06:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556d3b9cd270 con 0x556d3bacbcb0 2024-09-10T10:06:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.865+0000 7f6f47a3e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d3ba212b0 con 0x556d3bac0af0 2024-09-10T10:06:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bacb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bacb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39098/0 (socket says 172.21.15.45:39098) 2024-09-10T10:06:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3363974658 learned_addr learned my addr 172.21.15.45:0/3363974658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f2f7fe640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6f280a2e20 0x7f6f280a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bace0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3363974658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 msgr2=0x556d3bac0ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bac0ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3363974658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 msgr2=0x556d3bace0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bace0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3363974658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d3b9cc340 con 0x556d3bac1410 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f2f7fe640 1 -- 172.21.15.45:0/2668070187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f6f380ce920 con 0x7f6f4019ee00 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.866+0000 7f6f3ed9d640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6f280a2e20 0x7f6f280a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.867+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bacb770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009620 tx=0x7f6f30009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df99d82e5c2f7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.867+0000 7f6f2cff9640 1 -- 172.21.15.45:0/3363974658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3003e070 con 0x556d3bac1410 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.867+0000 7f6f3ed9d640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6f280a2e20 0x7f6f280a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f380cc1f0 tx=0x7f6f38014c20 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.867+0000 7f6f2cff9640 1 -- 172.21.15.45:0/3363974658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f300053c0 con 0x556d3bac1410 2024-09-10T10:06:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.867+0000 7f6f2cff9640 1 -- 172.21.15.45:0/3363974658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30035040 con 0x556d3bac1410 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.869+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3363974658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 msgr2=0x556d3bacb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.869+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bacb770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009620 tx=0x7f6f30009650 comp rx=0 tx=0).stop 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.869+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3363974658 shutdown_connections 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.869+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bace0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.869+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bacb770 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.869+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3363974658 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bac0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.870+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3363974658 >> 172.21.15.45:0/3363974658 conn(0x556d3babd060 msgr2=0x556d3babd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.870+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3363974658 shutdown_connections 2024-09-10T10:06:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.870+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3363974658 wait complete. 2024-09-10T10:06:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.871+0000 7f6f47a3e740 1 -- start start 2024-09-10T10:06:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.871+0000 7f6f47a3e740 1 Processor -- start 2024-09-10T10:06:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bac8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bac96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f47a3e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556d3bace880 con 0x556d3bac1410 2024-09-10T10:06:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39104/0 (socket says 172.21.15.45:39104) 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3727048506 learned_addr learned my addr 172.21.15.45:0/3727048506 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556d3b9ae290 con 0x556d3bacbcb0 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.872+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x556d3bacea00 con 0x556d3bac0af0 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bac96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3727048506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 msgr2=0x556d3bac8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bac8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 msgr2=0x556d3bac96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bac96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f3e59c640 1 -- 172.21.15.45:0/3727048506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3b738210 con 0x556d3bac1410 2024-09-10T10:06:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f47a3e740 10 client.0 _ll_register_callbacks cb 0x556d3b9df140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006010 tx=0x7f6f300058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd7f4ae38a319b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.873+0000 7f6f25ffb640 10 client.15348 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.874+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f300053c0 con 0x556d3bac1410 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.874+0000 7f6f25ffb640 1 -- 172.21.15.45:0/3727048506 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3004b030 con 0x556d3bac1410 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.874+0000 7f6f25ffb640 1 -- 172.21.15.45:0/3727048506 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f6f30031c00 con 0x556d3bac1410 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.874+0000 7f6f25ffb640 1 -- 172.21.15.45:0/3727048506 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6f30055050 con 0x556d3bac1410 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.874+0000 7f6f47a3e740 10 client.15348 fetch_fsmap learned FSMap version 19 2024-09-10T10:06:32.877 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.874+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=19}) -- 0x556d3b804620 con 0x556d3bac1410 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.875+0000 7f6f25ffb640 1 -- 172.21.15.45:0/3727048506 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6f30004020 con 0x556d3bac1410 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.875+0000 7f6f47a3e740 10 client.15348 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.875+0000 7f6f47a3e740 10 client.15348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.875+0000 7f6f47a3e740 20 client.15348 populate_metadata read hostname 'smithi045' 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.875+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3badb800 0x556d3baddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.876+0000 7f6f47a3e740 4 client.15348 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.876+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d3ba32a20 con 0x556d3badb800 2024-09-10T10:06:32.878 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.876+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3badb800 0x556d3baddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.880+0000 7f6f25ffb640 10 client.15348 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:32.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.880+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3badb800 0x556d3baddbf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60847633aec830a2 server_cookie=6724f26db8bdce32 in_seq=0 out_seq=0 2024-09-10T10:06:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.883+0000 7f6f25ffb640 1 -- 172.21.15.45:0/3727048506 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d3ba32a20 con 0x556d3badb800 2024-09-10T10:06:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.883+0000 7f6f25ffb640 10 client.15348 handle_command_reply: tid=0 2024-09-10T10:06:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 client.15348 shutdown 2024-09-10T10:06:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 msgr2=0x556d3bac9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6f30006010 tx=0x7f6f300058d0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 shutdown_connections 2024-09-10T10:06:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3badb800 0x556d3baddbf0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bac96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/3727048506 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bac8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f25ffb640 0 client.15348 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 >> 172.21.15.45:0/3727048506 conn(0x556d3babd060 msgr2=0x556d3bac2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 shutdown_connections 2024-09-10T10:06:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.884+0000 7f6f47a3e740 1 -- 172.21.15.45:0/3727048506 wait complete. 2024-09-10T10:06:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.885+0000 7f6f47a3e740 20 client.15348 trim_cache size 0 max 16384 2024-09-10T10:06:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.923+0000 7f6f47a3e740 1 Processor -- start 2024-09-10T10:06:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.924+0000 7f6f47a3e740 1 -- start start 2024-09-10T10:06:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.924+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bacabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.924+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.924+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bae3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.924+0000 7f6f47a3e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401ad9b0 con 0x556d3bac1410 2024-09-10T10:06:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.924+0000 7f6f47a3e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401b0250 con 0x556d3bacbcb0 2024-09-10T10:06:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.924+0000 7f6f47a3e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401ae040 con 0x556d3bac0af0 2024-09-10T10:06:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39110/0 (socket says 172.21.15.45:39110) 2024-09-10T10:06:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 -- 172.21.15.45:0/2687069076 learned_addr learned my addr 172.21.15.45:0/2687069076 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bae3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 -- 172.21.15.45:0/2687069076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 msgr2=0x556d3bacabc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bacabc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 -- 172.21.15.45:0/2687069076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 msgr2=0x556d3bae3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bae3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.925+0000 7f6f3e59c640 1 -- 172.21.15.45:0/2687069076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d3bac0870 con 0x556d3bac1410 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.926+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6f30005940 tx=0x7f6f30047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc08647825d98266 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.926+0000 7f6f27fff640 1 -- 172.21.15.45:0/2687069076 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30035070 con 0x556d3bac1410 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.926+0000 7f6f27fff640 1 -- 172.21.15.45:0/2687069076 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f300053c0 con 0x556d3bac1410 2024-09-10T10:06:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.926+0000 7f6f27fff640 1 -- 172.21.15.45:0/2687069076 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3005e070 con 0x556d3bac1410 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 -- 172.21.15.45:0/2687069076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 msgr2=0x556d3bac9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6f30005940 tx=0x7f6f30047000 comp rx=0 tx=0).stop 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 -- 172.21.15.45:0/2687069076 shutdown_connections 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacbcb0 0x556d3bae3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac1410 0x556d3bac9500 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/2687069076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bac0af0 0x556d3bacabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 -- 172.21.15.45:0/2687069076 >> 172.21.15.45:0/2687069076 conn(0x556d3babd060 msgr2=0x556d3baca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 -- 172.21.15.45:0/2687069076 shutdown_connections 2024-09-10T10:06:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.927+0000 7f6f47a3e740 1 -- 172.21.15.45:0/2687069076 wait complete. 2024-09-10T10:06:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.928+0000 7f6f47a3e740 1 -- start start 2024-09-10T10:06:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.928+0000 7f6f47a3e740 1 Processor -- start 2024-09-10T10:06:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.928+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac0af0 0x556d3bad3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.928+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bacbcb0 0x556d3bad44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.928+0000 7f6f3dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac0af0 0x556d3bad3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac0af0 0x556d3bad3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39124/0 (socket says 172.21.15.45:39124) 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.928+0000 7f6f47a3e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacf280 0x556d3bacf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 -- 172.21.15.45:0/1282499565 learned_addr learned my addr 172.21.15.45:0/1282499565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401ae710 con 0x556d3bac0af0 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401afb80 con 0x556d3bacf280 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6f401aede0 con 0x556d3bacbcb0 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f47a3e740 10 client.0 _ll_register_callbacks cb 0x7f6f30009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 -- 172.21.15.45:0/1282499565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bacbcb0 msgr2=0x556d3bad44c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bacbcb0 0x556d3bad44c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 -- 172.21.15.45:0/1282499565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacf280 msgr2=0x556d3bacf6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacf280 0x556d3bacf6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 -- 172.21.15.45:0/1282499565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3bacfef0 con 0x556d3bac0af0 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f300053c0 con 0x556d3bac0af0 2024-09-10T10:06:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f3dd9b640 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac0af0 0x556d3bad3f80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6f34001ba0 tx=0x7f6f340019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68c7b3427aa431e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.929+0000 7f6f25ffb640 10 client.15354 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:06:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f25ffb640 1 -- 172.21.15.45:0/1282499565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34006db0 con 0x556d3bac0af0 2024-09-10T10:06:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f25ffb640 1 -- 172.21.15.45:0/1282499565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f6f34007050 con 0x556d3bac0af0 2024-09-10T10:06:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f25ffb640 1 -- 172.21.15.45:0/1282499565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6f34007270 con 0x556d3bac0af0 2024-09-10T10:06:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 10 client.15354 fetch_fsmap learned FSMap version 19 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=19}) -- 0x556d3bb3f140 con 0x556d3bac0af0 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f25ffb640 1 -- 172.21.15.45:0/1282499565 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f6f34007410 con 0x556d3bac0af0 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 10 client.15354 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 20 client.15354 populate_metadata read hostname 'smithi045' 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3bb3f750 0x556d3bb41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 4 client.15354 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.930+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d3ba2cd70 con 0x556d3bb3f750 2024-09-10T10:06:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.931+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3bb3f750 0x556d3bb41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.935+0000 7f6f25ffb640 10 client.15354 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:32.935+0000 7f6f3e59c640 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3bb3f750 0x556d3bb41ba0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b470cfc3e36825 server_cookie=dd3478d8cb056d1e in_seq=0 out_seq=0 2024-09-10T10:06:33.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:33 smithi047 ceph-mon[35244]: pgmap v103: 97 pgs: 97 active+clean; 30 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 727 MiB/s wr, 2.10k op/s 2024-09-10T10:06:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:33 smithi162 ceph-mon[34963]: pgmap v103: 97 pgs: 97 active+clean; 30 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 727 MiB/s wr, 2.10k op/s 2024-09-10T10:06:33.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:33 smithi045 ceph-mon[31114]: pgmap v103: 97 pgs: 97 active+clean; 30 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 727 MiB/s wr, 2.10k op/s 2024-09-10T10:06:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.319+0000 7f6f25ffb640 1 -- 172.21.15.45:0/1282499565 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d3ba2cd70 con 0x556d3bb3f750 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.319+0000 7f6f25ffb640 10 client.15354 handle_command_reply: tid=0 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.319+0000 7f6f47a3e740 1 client.15354 shutdown 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac0af0 msgr2=0x556d3bad3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac0af0 0x556d3bad3f80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6f34001ba0 tx=0x7f6f340019f0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 shutdown_connections 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x556d3bb3f750 0x556d3bb41ba0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556d3bacf280 0x556d3bacf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x556d3bacbcb0 0x556d3bad44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 --2- 172.21.15.45:0/1282499565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556d3bac0af0 0x556d3bad3f80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 >> 172.21.15.45:0/1282499565 conn(0x556d3babd060 msgr2=0x556d3bac7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f25ffb640 0 client.15354 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 shutdown_connections 2024-09-10T10:06:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.320+0000 7f6f47a3e740 1 -- 172.21.15.45:0/1282499565 wait complete. 2024-09-10T10:06:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.322+0000 7f6f47a3e740 20 client.15354 trim_cache size 0 max 16384 2024-09-10T10:06:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:06:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:06:35.325 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:06:35.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.323+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6f280a2e20 msgr2=0x7f6f280a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6f280a2e20 0x7f6f280a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f380cc1f0 tx=0x7f6f38014c20 comp rx=0 tx=0).stop 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f4019ee00 msgr2=0x7f6f4019f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f4019ee00 0x7f6f4019f2b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800b990 tx=0x7f6f3800be50 comp rx=0 tx=0).stop 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 shutdown_connections 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6f4019ee00 0x7f6f4019f2b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f6f280a2e20 0x7f6f280a52e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4010a2a0 0x7f6f4019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.324+0000 7f6f44bc4640 1 --2- 172.21.15.45:0/2668070187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6f400ffdd0 0x7f6f4019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.325+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 >> 172.21.15.45:0/2668070187 conn(0x7f6f400fba30 msgr2=0x7f6f400fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.325+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 shutdown_connections 2024-09-10T10:06:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:35.325+0000 7f6f44bc4640 1 -- 172.21.15.45:0/2668070187 wait complete. 2024-09-10T10:06:35.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:35 smithi162 ceph-mon[34963]: pgmap v104: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 GiB/s wr, 2.37k op/s 2024-09-10T10:06:35.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:35 smithi045 ceph-mon[31114]: pgmap v104: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 GiB/s wr, 2.37k op/s 2024-09-10T10:06:35.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:35 smithi047 ceph-mon[35244]: pgmap v104: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 GiB/s wr, 2.37k op/s 2024-09-10T10:06:37.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:37 smithi162 ceph-mon[34963]: pgmap v105: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 754 MiB/s wr, 1.79k op/s 2024-09-10T10:06:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:37 smithi047 ceph-mon[35244]: pgmap v105: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 754 MiB/s wr, 1.79k op/s 2024-09-10T10:06:37.664 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:37 smithi045 ceph-mon[31114]: pgmap v105: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 754 MiB/s wr, 1.79k op/s 2024-09-10T10:06:39.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:39 smithi162 ceph-mon[34963]: pgmap v106: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 824 MiB/s wr, 1.98k op/s 2024-09-10T10:06:39.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:39 smithi045 ceph-mon[31114]: pgmap v106: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 824 MiB/s wr, 1.98k op/s 2024-09-10T10:06:39.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:39 smithi047 ceph-mon[35244]: pgmap v106: 97 pgs: 97 active+clean; 30 GiB data, 154 GiB used, 919 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 824 MiB/s wr, 1.98k op/s 2024-09-10T10:06:41.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:41 smithi045 ceph-mon[31114]: pgmap v107: 97 pgs: 97 active+clean; 30 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 964 MiB/s wr, 2.21k op/s 2024-09-10T10:06:41.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:41 smithi047 ceph-mon[35244]: pgmap v107: 97 pgs: 97 active+clean; 30 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 964 MiB/s wr, 2.21k op/s 2024-09-10T10:06:41.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:41 smithi162 ceph-mon[34963]: pgmap v107: 97 pgs: 97 active+clean; 30 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 964 MiB/s wr, 2.21k op/s 2024-09-10T10:06:43.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:43 smithi162 ceph-mon[34963]: pgmap v108: 97 pgs: 97 active+clean; 30 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 716 MiB/s wr, 1.25k op/s 2024-09-10T10:06:43.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:43 smithi045 ceph-mon[31114]: pgmap v108: 97 pgs: 97 active+clean; 30 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 716 MiB/s wr, 1.25k op/s 2024-09-10T10:06:43.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:43 smithi047 ceph-mon[35244]: pgmap v108: 97 pgs: 97 active+clean; 30 GiB data, 158 GiB used, 915 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 716 MiB/s wr, 1.25k op/s 2024-09-10T10:06:45.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:45 smithi162 ceph-mon[34963]: pgmap v109: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 GiB/s wr, 1.70k op/s 2024-09-10T10:06:45.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:45 smithi045 ceph-mon[31114]: pgmap v109: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 GiB/s wr, 1.70k op/s 2024-09-10T10:06:45.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:45 smithi047 ceph-mon[35244]: pgmap v109: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.0 GiB/s wr, 1.70k op/s 2024-09-10T10:06:47.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:47 smithi162 ceph-mon[34963]: pgmap v110: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 775 MiB/s wr, 1.53k op/s 2024-09-10T10:06:47.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:47 smithi045 ceph-mon[31114]: pgmap v110: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 775 MiB/s wr, 1.53k op/s 2024-09-10T10:06:47.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:47 smithi047 ceph-mon[35244]: pgmap v110: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 775 MiB/s wr, 1.53k op/s 2024-09-10T10:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.695+0000 7f3a11627640 1 -- 172.21.15.45:0/1333535096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 msgr2=0x7f3a0c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.695+0000 7f3a11627640 1 --2- 172.21.15.45:0/1333535096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 0x7f3a0c11d530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009930 tx=0x7f3a0002a460 comp rx=0 tx=0).stop 2024-09-10T10:06:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.698+0000 7f3a11627640 1 -- 172.21.15.45:0/1333535096 shutdown_connections 2024-09-10T10:06:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.698+0000 7f3a11627640 1 --2- 172.21.15.45:0/1333535096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 0x7f3a0c11d530 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.698+0000 7f3a11627640 1 --2- 172.21.15.45:0/1333535096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a0c1095d0 0x7f3a0c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.698+0000 7f3a11627640 1 --2- 172.21.15.45:0/1333535096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1091f0 0x7f3a0c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.698+0000 7f3a11627640 1 -- 172.21.15.45:0/1333535096 >> 172.21.15.45:0/1333535096 conn(0x7f3a0c06ce00 msgr2=0x7f3a0c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.698+0000 7f3a11627640 1 -- 172.21.15.45:0/1333535096 shutdown_connections 2024-09-10T10:06:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.698+0000 7f3a11627640 1 -- 172.21.15.45:0/1333535096 wait complete. 2024-09-10T10:06:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.699+0000 7f3a11627640 1 Processor -- start 2024-09-10T10:06:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.700+0000 7f3a11627640 1 -- start start 2024-09-10T10:06:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.700+0000 7f3a11627640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1091f0 0x7f3a0c118540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.700+0000 7f3a11627640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a0c1095d0 0x7f3a0c118a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.700+0000 7f3a11627640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 0x7f3a0c111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.700+0000 7f3a11627640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c120dd0 con 0x7f3a0c1095d0 2024-09-10T10:06:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.700+0000 7f3a11627640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c120c50 con 0x7f3a0c1091f0 2024-09-10T10:06:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.700+0000 7f3a11627640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c120f50 con 0x7f3a0c11b140 2024-09-10T10:06:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.702+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a0c1095d0 0x7f3a0c118a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.702+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a0c1095d0 0x7f3a0c118a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47134/0 (socket says 172.21.15.45:47134) 2024-09-10T10:06:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.702+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/725490062 learned_addr learned my addr 172.21.15.45:0/725490062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.702+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 0x7f3a0c111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.704+0000 7f3a0b7fe640 1 -- 172.21.15.45:0/725490062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1091f0 msgr2=0x7f3a0c118540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:06:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.704+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1091f0 0x7f3a0c118540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.704+0000 7f3a0b7fe640 1 -- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a0c1095d0 msgr2=0x7f3a0c118a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.704+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a0c1095d0 0x7f3a0c118a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.704+0000 7f3a0b7fe640 1 -- 172.21.15.45:0/725490062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c111f00 con 0x7f3a0c11b140 2024-09-10T10:06:48.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.711+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 0x7f3a0c111640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a0002aa50 tx=0x7f3a00039520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.772+0000 7f39f3fff640 1 -- 172.21.15.45:0/725490062 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00040030 con 0x7f3a0c11b140 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.772+0000 7f39f3fff640 1 -- 172.21.15.45:0/725490062 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a00030e40 con 0x7f3a0c11b140 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.772+0000 7f39f3fff640 1 -- 172.21.15.45:0/725490062 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00035070 con 0x7f3a0c11b140 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.772+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c112190 con 0x7f3a0c11b140 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.772+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c116240 con 0x7f3a0c11b140 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.777+0000 7f3a143c1740 1 Processor -- start 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.778+0000 7f3a143c1740 1 -- start start 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.778+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcb7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.778+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babce4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.778+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.778+0000 7f3a143c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557babc1c2f0 con 0x557babcb7af0 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.778+0000 7f3a143c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557babbc4270 con 0x557babce4cc0 2024-09-10T10:06:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.778+0000 7f3a143c1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557babc182b0 con 0x557babcb8410 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.779+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.779+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:58756/0 (socket says 172.21.15.45:58756) 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.779+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/2718326045 learned_addr learned my addr 172.21.15.45:0/2718326045 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.780+0000 7f3a0affd640 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babce4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.780+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/2718326045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 msgr2=0x557babce4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.780+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babce4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.780+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/2718326045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 msgr2=0x557babcb7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.780+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcb7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.780+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/2718326045 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557babbc3340 con 0x557babce4cc0 2024-09-10T10:06:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.781+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce70b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009f80 tx=0x7f3a04008910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17032f85483d6659 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.785+0000 7f39f17fa640 1 -- 172.21.15.45:0/2718326045 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04012510 con 0x557babce4cc0 2024-09-10T10:06:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.785+0000 7f39f17fa640 1 -- 172.21.15.45:0/2718326045 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04012b90 con 0x557babce4cc0 2024-09-10T10:06:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.786+0000 7f3a143c1740 1 -- 172.21.15.45:0/2718326045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 msgr2=0x557babce70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.786+0000 7f3a143c1740 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce70b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009f80 tx=0x7f3a04008910 comp rx=0 tx=0).stop 2024-09-10T10:06:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.787+0000 7f3a143c1740 1 -- 172.21.15.45:0/2718326045 shutdown_connections 2024-09-10T10:06:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.787+0000 7f3a143c1740 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce70b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.787+0000 7f3a143c1740 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babce4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.787+0000 7f3a143c1740 1 --2- 172.21.15.45:0/2718326045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcb7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.787+0000 7f3a143c1740 1 -- 172.21.15.45:0/2718326045 >> 172.21.15.45:0/2718326045 conn(0x557babcb4060 msgr2=0x557babcb4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.789+0000 7f3a143c1740 1 -- 172.21.15.45:0/2718326045 shutdown_connections 2024-09-10T10:06:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.790+0000 7f3a143c1740 1 -- 172.21.15.45:0/2718326045 wait complete. 2024-09-10T10:06:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.791+0000 7f3a143c1740 1 -- start start 2024-09-10T10:06:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.791+0000 7f3a143c1740 1 Processor -- start 2024-09-10T10:06:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.792+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb7af0 0x557babce1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.792+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb8410 0x557babce2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.792+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.792+0000 7f3a143c1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557babce78e0 con 0x557babcb8410 2024-09-10T10:06:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.792+0000 7f3a143c1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557babba5290 con 0x557babce4cc0 2024-09-10T10:06:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.792+0000 7f3a143c1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557babce7a60 con 0x557babcb7af0 2024-09-10T10:06:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.792+0000 7f3a143c1740 10 client.0 _ll_register_callbacks cb 0x557babbd6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.795+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb7af0 0x557babce1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.795+0000 7f3a0a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb7af0 0x557babce1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:32974/0 (socket says 172.21.15.45:32974) 2024-09-10T10:06:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.795+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/1566150295 learned_addr learned my addr 172.21.15.45:0/1566150295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:48.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.795+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.795+0000 7f3a0affd640 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb8410 0x557babce2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.801+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 msgr2=0x557babce2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.801+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.801+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/1566150295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb8410 msgr2=0x557babce2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.801+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb8410 0x557babce2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.801+0000 7f3a0a7fc640 1 -- 172.21.15.45:0/1566150295 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bab92f210 con 0x557babcb7af0 2024-09-10T10:06:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.806+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb7af0 0x557babce1b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a04016870 tx=0x7f3a04016970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12730fbbb65014ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.806+0000 7f39eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:06:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.860+0000 7f39eeff5640 1 -- 172.21.15.45:0/1566150295 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0402a060 con 0x557babcb7af0 2024-09-10T10:06:48.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.860+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a04012b90 con 0x557babcb7af0 2024-09-10T10:06:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.863+0000 7f39f3fff640 1 -- 172.21.15.45:0/725490062 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3a00043030 con 0x7f3a0c11b140 2024-09-10T10:06:48.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.863+0000 7f39eeff5640 1 -- 172.21.15.45:0/1566150295 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f3a0401c040 con 0x557babcb7af0 2024-09-10T10:06:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.866+0000 7f39f3fff640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f39f40a2db0 0x7f39f40a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.867+0000 7f39f3fff640 1 -- 172.21.15.45:0/725490062 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f3a000efb70 con 0x7f3a0c11b140 2024-09-10T10:06:48.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.868+0000 7f39eeff5640 1 -- 172.21.15.45:0/1566150295 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3a04012950 con 0x557babcb7af0 2024-09-10T10:06:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.868+0000 7f3a143c1740 10 client.24970 fetch_fsmap learned FSMap version 19 2024-09-10T10:06:48.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.868+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x557bab9fb620 con 0x557babcb7af0 2024-09-10T10:06:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.871+0000 7f3a0affd640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f39f40a2db0 0x7f39f40a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.872+0000 7f39eeff5640 1 -- 172.21.15.45:0/1566150295 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a04012b90 con 0x557babcb7af0 2024-09-10T10:06:48.879 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.875+0000 7f39eeff5640 1 -- 172.21.15.45:0/1566150295 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f3a04024450 con 0x557babcb7af0 2024-09-10T10:06:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.877+0000 7f3a143c1740 10 client.24970 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:06:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.877+0000 7f3a143c1740 10 client.24970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:06:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.877+0000 7f3a143c1740 20 client.24970 populate_metadata read hostname 'smithi045' 2024-09-10T10:06:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.877+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babcf4ba0 0x557babcf6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.877+0000 7f3a143c1740 4 client.24970 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:06:48.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.877+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557babc29a20 con 0x557babcf4ba0 2024-09-10T10:06:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.878+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babcf4ba0 0x557babcf6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.880+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babcf4ba0 0x557babcf6f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afcb41fad3874511 server_cookie=d08662898b483ea1 in_seq=0 out_seq=0 2024-09-10T10:06:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.880+0000 7f3a0affd640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f39f40a2db0 0x7f39f40a5270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x557babce30c0 tx=0x7f39fc008040 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:48.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.880+0000 7f39eeff5640 10 client.24970 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f39eeff5640 1 -- 172.21.15.45:0/1566150295 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557babc29a20 con 0x557babcf4ba0 2024-09-10T10:06:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f39eeff5640 10 client.24970 handle_command_reply: tid=0 2024-09-10T10:06:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 client.24970 shutdown 2024-09-10T10:06:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb7af0 msgr2=0x557babce1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:48.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb7af0 0x557babce1b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a04016870 tx=0x7f3a04016970 comp rx=0 tx=0).stop 2024-09-10T10:06:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 shutdown_connections 2024-09-10T10:06:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babcf4ba0 0x557babcf6f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babce4cc0 0x557babce2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb8410 0x557babce2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1566150295 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb7af0 0x557babce1b10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.885+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 >> 172.21.15.45:0/1566150295 conn(0x557babcb4060 msgr2=0x557babcb9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.886+0000 7f39eeff5640 0 client.24970 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.886+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 shutdown_connections 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.886+0000 7f3a143c1740 1 -- 172.21.15.45:0/1566150295 wait complete. 2024-09-10T10:06:48.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.886+0000 7f3a143c1740 20 client.24970 trim_cache size 0 max 16384 2024-09-10T10:06:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.933+0000 7f3a143c1740 1 Processor -- start 2024-09-10T10:06:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.933+0000 7f3a143c1740 1 -- start start 2024-09-10T10:06:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.934+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.935+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb8410 0x557babcb4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.935+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.935+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcba5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47164/0 (socket says 172.21.15.45:47164) 2024-09-10T10:06:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.936+0000 7f3a0affd640 1 -- 172.21.15.45:0/1510256778 learned_addr learned my addr 172.21.15.45:0/1510256778 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:48.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.936+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babce4cc0 0x557babce23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.936+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39f800c870 con 0x557babcb7af0 2024-09-10T10:06:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.936+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb8410 0x557babcb4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.936+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f39f800f110 con 0x557babcb8410 2024-09-10T10:06:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.939+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babce4cc0 0x557babce23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.940+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f39f800cf00 con 0x557babce4cc0 2024-09-10T10:06:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.942+0000 7f3a0b7fe640 1 -- 172.21.15.45:0/1510256778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb8410 msgr2=0x557babcb4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.942+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb8410 0x557babcb4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.942+0000 7f3a0b7fe640 1 -- 172.21.15.45:0/1510256778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 msgr2=0x557babcba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.942+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.942+0000 7f3a0b7fe640 1 -- 172.21.15.45:0/1510256778 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557babcb8240 con 0x557babce4cc0 2024-09-10T10:06:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.943+0000 7f3a0b7fe640 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babce4cc0 0x557babce23c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3a000ed260 tx=0x7f3a000bba80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb7bbd7aa3bdd61f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.943+0000 7f39f0ff9640 1 -- 172.21.15.45:0/1510256778 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a000bc060 con 0x557babce4cc0 2024-09-10T10:06:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.943+0000 7f39f0ff9640 1 -- 172.21.15.45:0/1510256778 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a000c0410 con 0x557babce4cc0 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.945+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babce4cc0 msgr2=0x557babce23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.945+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babce4cc0 0x557babce23c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3a000ed260 tx=0x7f3a000bba80 comp rx=0 tx=0).stop 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.945+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 shutdown_connections 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.945+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babce4cc0 0x557babce23c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.945+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb8410 0x557babcb4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.946+0000 7f3a143c1740 1 --2- 172.21.15.45:0/1510256778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcb7af0 0x557babcba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.946+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 >> 172.21.15.45:0/1510256778 conn(0x557babcb4060 msgr2=0x557babce3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.946+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 shutdown_connections 2024-09-10T10:06:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.946+0000 7f3a143c1740 1 -- 172.21.15.45:0/1510256778 wait complete. 2024-09-10T10:06:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.947+0000 7f3a143c1740 1 -- start start 2024-09-10T10:06:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.947+0000 7f3a143c1740 1 Processor -- start 2024-09-10T10:06:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.948+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb7af0 0x557babced0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.948+0000 7f3a143c1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babced5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.949+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb7af0 0x557babced0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.949+0000 7f3a0affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb7af0 0x557babced0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:58788/0 (socket says 172.21.15.45:58788) 2024-09-10T10:06:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.949+0000 7f3a0affd640 1 -- 172.21.15.45:0/152700697 learned_addr learned my addr 172.21.15.45:0/152700697 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:06:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.949+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babced5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.950+0000 7f3a143c1740 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcbc370 0x557babcbc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.950+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f39f800d5d0 con 0x557babcbc370 2024-09-10T10:06:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.950+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f39f800ea40 con 0x557babcb7af0 2024-09-10T10:06:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.950+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f39f800dca0 con 0x557babcb8410 2024-09-10T10:06:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a143c1740 10 client.0 _ll_register_callbacks cb 0x7f3a04016480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:06:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a0affd640 1 -- 172.21.15.45:0/152700697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 msgr2=0x557babced5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a0affd640 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babced5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a0affd640 1 -- 172.21.15.45:0/152700697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcbc370 msgr2=0x557babcbc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:06:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a0affd640 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcbc370 0x557babcbc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:48.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a0affd640 1 -- 172.21.15.45:0/152700697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557babcbcfd0 con 0x557babcb7af0 2024-09-10T10:06:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557babc23d70 con 0x557babcb7af0 2024-09-10T10:06:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.951+0000 7f3a0affd640 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb7af0 0x557babced0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0157a0 tx=0x7f39fc0046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1cc42956b2189e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:06:48.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.954+0000 7f39eeff5640 10 client.25061 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:06:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.958+0000 7f39eeff5640 1 -- 172.21.15.45:0/152700697 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc02f050 con 0x557babcb7af0 2024-09-10T10:06:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.958+0000 7f39eeff5640 1 -- 172.21.15.45:0/152700697 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f39fc013b20 con 0x557babcb7af0 2024-09-10T10:06:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.958+0000 7f39eeff5640 1 -- 172.21.15.45:0/152700697 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f39fc025070 con 0x557babcb7af0 2024-09-10T10:06:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.958+0000 7f3a143c1740 10 client.25061 fetch_fsmap learned FSMap version 19 2024-09-10T10:06:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.958+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=19}) -- 0x557babd5f530 con 0x557babcb7af0 2024-09-10T10:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.960+0000 7f39eeff5640 1 -- 172.21.15.45:0/152700697 <== mon.2 v2:172.21.15.47:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f39fc01e050 con 0x557babcb7af0 2024-09-10T10:06:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.960+0000 7f3a143c1740 10 client.25061 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:06:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.960+0000 7f3a143c1740 10 client.25061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:06:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.960+0000 7f3a143c1740 20 client.25061 populate_metadata read hostname 'smithi045' 2024-09-10T10:06:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.960+0000 7f3a143c1740 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babd5fcb0 0x557babd62150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:06:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.960+0000 7f3a143c1740 4 client.25061 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:06:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.960+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557babc23d70 con 0x557babd5fcb0 2024-09-10T10:06:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.961+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babd5fcb0 0x557babd62150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:06:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.962+0000 7f3a0a7fc640 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babd5fcb0 0x557babd62150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e336f2cecc27a3a server_cookie=cd8ecc66fde32e1d in_seq=0 out_seq=0 2024-09-10T10:06:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:48.963+0000 7f39eeff5640 10 client.25061 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:49.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:48 smithi162 ceph-mon[34963]: pgmap v111: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 848 MiB/s wr, 1.68k op/s 2024-09-10T10:06:49.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:48 smithi045 ceph-mon[31114]: pgmap v111: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 848 MiB/s wr, 1.68k op/s 2024-09-10T10:06:49.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:48 smithi047 ceph-mon[35244]: pgmap v111: 97 pgs: 97 active+clean; 30 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 848 MiB/s wr, 1.68k op/s 2024-09-10T10:06:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:06:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:06:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:06:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:06:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f39eeff5640 1 -- 172.21.15.45:0/152700697 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557babc23d70 con 0x557babd5fcb0 2024-09-10T10:06:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f39eeff5640 10 client.25061 handle_command_reply: tid=0 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 client.25061 shutdown 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb7af0 msgr2=0x557babced0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb7af0 0x557babced0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0157a0 tx=0x7f39fc0046e0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 shutdown_connections 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557babd5fcb0 0x557babd62150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557babcbc370 0x557babcbc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557babcb8410 0x557babced5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 --2- 172.21.15.45:0/152700697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557babcb7af0 0x557babced0a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 >> 172.21.15.45:0/152700697 conn(0x557babcb4060 msgr2=0x557babce3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f39eeff5640 0 client.25061 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 shutdown_connections 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.960+0000 7f3a143c1740 1 -- 172.21.15.45:0/152700697 wait complete. 2024-09-10T10:06:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.961+0000 7f3a143c1740 20 client.25061 trim_cache size 0 max 16384 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f39f40a2db0 msgr2=0x7f39f40a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f39f40a2db0 0x7f39f40a5270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x557babce30c0 tx=0x7f39fc008040 comp rx=0 tx=0).stop 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 msgr2=0x7f3a0c111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 0x7f3a0c111640 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a0002aa50 tx=0x7f3a00039520 comp rx=0 tx=0).stop 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 shutdown_connections 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a0c11b140 0x7f3a0c111640 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a0c1095d0 0x7f3a0c118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f39f40a2db0 0x7f39f40a5270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 --2- 172.21.15.45:0/725490062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c1091f0 0x7f3a0c118540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.963+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 >> 172.21.15.45:0/725490062 conn(0x7f3a0c06ce00 msgr2=0x7f3a0c11b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.964+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 shutdown_connections 2024-09-10T10:06:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:06:50.964+0000 7f3a11627640 1 -- 172.21.15.45:0/725490062 wait complete. 2024-09-10T10:06:51.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:50 smithi162 ceph-mon[34963]: pgmap v112: 97 pgs: 97 active+clean; 30 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 963 MiB/s wr, 1.75k op/s 2024-09-10T10:06:51.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:50 smithi045 ceph-mon[31114]: pgmap v112: 97 pgs: 97 active+clean; 30 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 963 MiB/s wr, 1.75k op/s 2024-09-10T10:06:51.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:50 smithi047 ceph-mon[35244]: pgmap v112: 97 pgs: 97 active+clean; 30 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 963 MiB/s wr, 1.75k op/s 2024-09-10T10:06:53.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:53 smithi162 ceph-mon[34963]: pgmap v113: 97 pgs: 97 active+clean; 30 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 708 MiB/s wr, 1.10k op/s 2024-09-10T10:06:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:52 smithi045 ceph-mon[31114]: pgmap v113: 97 pgs: 97 active+clean; 30 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 708 MiB/s wr, 1.10k op/s 2024-09-10T10:06:53.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:52 smithi047 ceph-mon[35244]: pgmap v113: 97 pgs: 97 active+clean; 30 GiB data, 167 GiB used, 906 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 708 MiB/s wr, 1.10k op/s 2024-09-10T10:06:55.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:55 smithi162 ceph-mon[34963]: pgmap v114: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1016 MiB/s wr, 1.83k op/s 2024-09-10T10:06:55.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:55 smithi045 ceph-mon[31114]: pgmap v114: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1016 MiB/s wr, 1.83k op/s 2024-09-10T10:06:55.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:55 smithi047 ceph-mon[35244]: pgmap v114: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 1016 MiB/s wr, 1.83k op/s 2024-09-10T10:06:57.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:57 smithi162 ceph-mon[34963]: pgmap v115: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 748 MiB/s wr, 1.44k op/s 2024-09-10T10:06:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:57 smithi045 ceph-mon[31114]: pgmap v115: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 748 MiB/s wr, 1.44k op/s 2024-09-10T10:06:57.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:57 smithi047 ceph-mon[35244]: pgmap v115: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 748 MiB/s wr, 1.44k op/s 2024-09-10T10:06:59.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:06:59 smithi045 ceph-mon[31114]: pgmap v116: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 808 MiB/s wr, 1.62k op/s 2024-09-10T10:06:59.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:06:59 smithi047 ceph-mon[35244]: pgmap v116: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 808 MiB/s wr, 1.62k op/s 2024-09-10T10:06:59.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:06:59 smithi162 ceph-mon[34963]: pgmap v116: 97 pgs: 97 active+clean; 30 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 808 MiB/s wr, 1.62k op/s 2024-09-10T10:07:01.701 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:01 smithi162 ceph-mon[34963]: pgmap v117: 97 pgs: 97 active+clean; 30 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 930 MiB/s wr, 2.13k op/s 2024-09-10T10:07:01.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:01 smithi045 ceph-mon[31114]: pgmap v117: 97 pgs: 97 active+clean; 30 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 930 MiB/s wr, 2.13k op/s 2024-09-10T10:07:01.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:01 smithi047 ceph-mon[35244]: pgmap v117: 97 pgs: 97 active+clean; 30 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 930 MiB/s wr, 2.13k op/s 2024-09-10T10:07:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:03 smithi162 ceph-mon[34963]: pgmap v118: 97 pgs: 97 active+clean; 30 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 697 MiB/s wr, 1.76k op/s 2024-09-10T10:07:03.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:03 smithi045 ceph-mon[31114]: pgmap v118: 97 pgs: 97 active+clean; 30 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 697 MiB/s wr, 1.76k op/s 2024-09-10T10:07:03.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:03 smithi047 ceph-mon[35244]: pgmap v118: 97 pgs: 97 active+clean; 30 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 697 MiB/s wr, 1.76k op/s 2024-09-10T10:07:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.209+0000 7f85b21f1640 1 -- 172.21.15.45:0/3260447421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac109b10 msgr2=0x7f85ac114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:04.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.209+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3260447421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac109b10 0x7f85ac114570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009c10 tx=0x7f85a402a710 comp rx=0 tx=0).stop 2024-09-10T10:07:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.216+0000 7f85b21f1640 1 -- 172.21.15.45:0/3260447421 shutdown_connections 2024-09-10T10:07:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.216+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3260447421 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac114ab0 0x7f85ac116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.216+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3260447421 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac109b10 0x7f85ac114570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.216+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3260447421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 0x7f85ac1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.216+0000 7f85b21f1640 1 -- 172.21.15.45:0/3260447421 >> 172.21.15.45:0/3260447421 conn(0x7f85ac06cbf0 msgr2=0x7f85ac06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:04.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.219+0000 7f85b21f1640 1 -- 172.21.15.45:0/3260447421 shutdown_connections 2024-09-10T10:07:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.221+0000 7f85b21f1640 1 -- 172.21.15.45:0/3260447421 wait complete. 2024-09-10T10:07:04.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.223+0000 7f85b21f1640 1 Processor -- start 2024-09-10T10:07:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.223+0000 7f85b21f1640 1 -- start start 2024-09-10T10:07:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.223+0000 7f85b21f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 0x7f85ac113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.224+0000 7f85b21f1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac109b10 0x7f85ac114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.224+0000 7f85b21f1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac114ab0 0x7f85ac10b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.224+0000 7f85b21f1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac11a4e0 con 0x7f85ac1091f0 2024-09-10T10:07:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.224+0000 7f85b21f1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac11a360 con 0x7f85ac114ab0 2024-09-10T10:07:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.224+0000 7f85b21f1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac11a660 con 0x7f85ac109b10 2024-09-10T10:07:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.226+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 0x7f85ac113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.226+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 0x7f85ac113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48758/0 (socket says 172.21.15.45:48758) 2024-09-10T10:07:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.226+0000 7f85b11ef640 1 -- 172.21.15.45:0/3817912786 learned_addr learned my addr 172.21.15.45:0/3817912786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.228+0000 7f85b11ef640 1 -- 172.21.15.45:0/3817912786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac109b10 msgr2=0x7f85ac114530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:07:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.228+0000 7f85b11ef640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac109b10 0x7f85ac114530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.228+0000 7f85b11ef640 1 -- 172.21.15.45:0/3817912786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac114ab0 msgr2=0x7f85ac10b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.228+0000 7f85b11ef640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac114ab0 0x7f85ac10b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.228+0000 7f85b11ef640 1 -- 172.21.15.45:0/3817912786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac10ba90 con 0x7f85ac1091f0 2024-09-10T10:07:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.230+0000 7f85b11ef640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 0x7f85ac113ff0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d950 tx=0x7f859c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.231+0000 7f85a27fc640 1 -- 172.21.15.45:0/3817912786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c017070 con 0x7f85ac1091f0 2024-09-10T10:07:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.231+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac10bd80 con 0x7f85ac1091f0 2024-09-10T10:07:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.231+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac10fd20 con 0x7f85ac1091f0 2024-09-10T10:07:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.232+0000 7f85a27fc640 1 -- 172.21.15.45:0/3817912786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c009c30 con 0x7f85ac1091f0 2024-09-10T10:07:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.233+0000 7f85a27fc640 1 -- 172.21.15.45:0/3817912786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c020060 con 0x7f85ac1091f0 2024-09-10T10:07:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.239+0000 7f85a27fc640 1 -- 172.21.15.45:0/3817912786 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f859c00f600 con 0x7f85ac1091f0 2024-09-10T10:07:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.241+0000 7f85b61ee740 1 Processor -- start 2024-09-10T10:07:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.243+0000 7f85b61ee740 1 -- start start 2024-09-10T10:07:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.244+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e79f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.244+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.244+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7ac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.244+0000 7f85b61ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55db8e7032f0 con 0x55db8e7a9e80 2024-09-10T10:07:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.244+0000 7f85b61ee740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55db8e6ab270 con 0x55db8e79f5e0 2024-09-10T10:07:04.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.244+0000 7f85b61ee740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55db8e6ff2b0 con 0x55db8e79ecc0 2024-09-10T10:07:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.245+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.245+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:60070/0 (socket says 172.21.15.45:60070) 2024-09-10T10:07:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.245+0000 7f85b11ef640 1 -- 172.21.15.45:0/3658135132 learned_addr learned my addr 172.21.15.45:0/3658135132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.245+0000 7f85a27fc640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f85980a2d30 0x7f85980a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.245+0000 7f85b19f0640 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e79f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.247+0000 7f85b11ef640 1 -- 172.21.15.45:0/3658135132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 msgr2=0x55db8e79f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.247+0000 7f85b11ef640 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e79f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.247+0000 7f85b11ef640 1 -- 172.21.15.45:0/3658135132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 msgr2=0x55db8e7ac270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:07:04.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.247+0000 7f85b11ef640 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7ac270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.247+0000 7f85b11ef640 1 -- 172.21.15.45:0/3658135132 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8e6aa340 con 0x55db8e79f5e0 2024-09-10T10:07:04.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.247+0000 7f85a27fc640 1 -- 172.21.15.45:0/3817912786 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f859c0104c0 con 0x7f85ac1091f0 2024-09-10T10:07:04.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.249+0000 7f85b11ef640 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7a9940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ccd10 tx=0x7f859c01ade0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b535db7c70e4ee8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.255+0000 7f85b19f0640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f85980a2d30 0x7f85980a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.258+0000 7f85977fe640 1 -- 172.21.15.45:0/3658135132 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c09c280 con 0x55db8e79f5e0 2024-09-10T10:07:04.260 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.258+0000 7f85977fe640 1 -- 172.21.15.45:0/3658135132 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c09ce40 con 0x55db8e79f5e0 2024-09-10T10:07:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.260+0000 7f85b61ee740 1 -- 172.21.15.45:0/3658135132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 msgr2=0x55db8e7a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:04.261 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.260+0000 7f85b61ee740 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7a9940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ccd10 tx=0x7f859c01ade0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.261+0000 7f85b19f0640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f85980a2d30 0x7f85980a51f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85a8005fd0 tx=0x7f85a80046c0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.261+0000 7f85b61ee740 1 -- 172.21.15.45:0/3658135132 shutdown_connections 2024-09-10T10:07:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.261+0000 7f85b61ee740 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.261+0000 7f85b61ee740 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7a9940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.261+0000 7f85b61ee740 1 --2- 172.21.15.45:0/3658135132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e79f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.261+0000 7f85b61ee740 1 -- 172.21.15.45:0/3658135132 >> 172.21.15.45:0/3658135132 conn(0x55db8e79b230 msgr2=0x55db8e79b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.264+0000 7f85b61ee740 1 -- 172.21.15.45:0/3658135132 shutdown_connections 2024-09-10T10:07:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.265+0000 7f85b61ee740 1 -- 172.21.15.45:0/3658135132 wait complete. 2024-09-10T10:07:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.266+0000 7f85b61ee740 1 -- start start 2024-09-10T10:07:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.266+0000 7f85b61ee740 1 Processor -- start 2024-09-10T10:07:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.268+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e79ecc0 0x55db8e7a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.268+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79f5e0 0x55db8e7a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.271 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.268+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e7a9e80 0x55db8e7a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.269+0000 7f85b61ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55db8e7ac9f0 con 0x55db8e79ecc0 2024-09-10T10:07:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.269+0000 7f85b61ee740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55db8e68c290 con 0x55db8e7a9e80 2024-09-10T10:07:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.269+0000 7f85b61ee740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55db8e7acb70 con 0x55db8e79f5e0 2024-09-10T10:07:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.269+0000 7f85b61ee740 10 client.0 _ll_register_callbacks cb 0x55db8e6bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:04.272 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.270+0000 7f85b19f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e7a9e80 0x55db8e7a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.270+0000 7f85b19f0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e7a9e80 0x55db8e7a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:60080/0 (socket says 172.21.15.45:60080) 2024-09-10T10:07:04.273 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.270+0000 7f85b19f0640 1 -- 172.21.15.45:0/4205084916 learned_addr learned my addr 172.21.15.45:0/4205084916 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.272+0000 7f85b19f0640 1 -- 172.21.15.45:0/4205084916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79f5e0 msgr2=0x55db8e7a73d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:07:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.272+0000 7f85b19f0640 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79f5e0 0x55db8e7a73d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.272+0000 7f85b19f0640 1 -- 172.21.15.45:0/4205084916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e79ecc0 msgr2=0x55db8e7a6e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:07:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.272+0000 7f85b19f0640 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e79ecc0 0x55db8e7a6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.272+0000 7f85b19f0640 1 -- 172.21.15.45:0/4205084916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8e416210 con 0x55db8e7a9e80 2024-09-10T10:07:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.275+0000 7f85b19f0640 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e7a9e80 0x55db8e7a7910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85a80136e0 tx=0x7f85a8005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc43d90bc3b5e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.277+0000 7f8594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:07:04.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.277+0000 7f8594ff9640 1 -- 172.21.15.45:0/4205084916 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a800f990 con 0x55db8e7a9e80 2024-09-10T10:07:04.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.279+0000 7f8594ff9640 1 -- 172.21.15.45:0/4205084916 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f85a800fb30 con 0x55db8e7a9e80 2024-09-10T10:07:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.279+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db8e710a20 con 0x55db8e7a9e80 2024-09-10T10:07:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.281+0000 7f8594ff9640 1 -- 172.21.15.45:0/4205084916 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f85a800fd30 con 0x55db8e7a9e80 2024-09-10T10:07:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.281+0000 7f85b61ee740 10 client.25070 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:04.283 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.281+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55db8e4e2620 con 0x55db8e7a9e80 2024-09-10T10:07:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.287+0000 7f8594ff9640 1 -- 172.21.15.45:0/4205084916 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55db8e710a20 con 0x55db8e7a9e80 2024-09-10T10:07:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.287+0000 7f8594ff9640 1 -- 172.21.15.45:0/4205084916 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f85a801c5a0 con 0x55db8e7a9e80 2024-09-10T10:07:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.287+0000 7f85b61ee740 10 client.25070 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.287+0000 7f85b61ee740 10 client.25070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.287+0000 7f85b61ee740 20 client.25070 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.287+0000 7f85b61ee740 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e7ba470 0x55db8e7bc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.288+0000 7f85b61ee740 4 client.25070 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:07:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.288+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db8e70ad70 con 0x55db8e7ba470 2024-09-10T10:07:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.290+0000 7f85b11ef640 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e7ba470 0x55db8e7bc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.292+0000 7f85b11ef640 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e7ba470 0x55db8e7bc930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe9523a168936e3 server_cookie=abe9e89563a4793f in_seq=0 out_seq=0 2024-09-10T10:07:04.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.293+0000 7f8594ff9640 10 client.25070 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.297+0000 7f8594ff9640 1 -- 172.21.15.45:0/4205084916 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db8e70ad70 con 0x55db8e7ba470 2024-09-10T10:07:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.297+0000 7f8594ff9640 10 client.25070 handle_command_reply: tid=0 2024-09-10T10:07:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.297+0000 7f85b61ee740 1 client.25070 shutdown 2024-09-10T10:07:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e7a9e80 msgr2=0x55db8e7a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e7a9e80 0x55db8e7a7910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85a80136e0 tx=0x7f85a8005c90 comp rx=0 tx=0).stop 2024-09-10T10:07:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 shutdown_connections 2024-09-10T10:07:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e7ba470 0x55db8e7bc930 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e7a9e80 0x55db8e7a7910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79f5e0 0x55db8e7a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 --2- 172.21.15.45:0/4205084916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e79ecc0 0x55db8e7a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.301+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 >> 172.21.15.45:0/4205084916 conn(0x55db8e79b230 msgr2=0x55db8e7a0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.302+0000 7f8594ff9640 0 client.25070 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.303+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 shutdown_connections 2024-09-10T10:07:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.303+0000 7f85b61ee740 1 -- 172.21.15.45:0/4205084916 wait complete. 2024-09-10T10:07:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.303+0000 7f85b61ee740 20 client.25070 trim_cache size 0 max 16384 2024-09-10T10:07:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.346+0000 7f85b61ee740 1 Processor -- start 2024-09-10T10:07:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.346+0000 7f85b61ee740 1 -- start start 2024-09-10T10:07:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.347+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.348+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e79a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.348+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:60330/0 (socket says 172.21.15.45:60330) 2024-09-10T10:07:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b11ef640 1 -- 172.21.15.45:0/1025583219 learned_addr learned my addr 172.21.15.45:0/1025583219 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:04.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b09ee640 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e79a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b19f0640 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f859000c860 con 0x55db8e7a9e80 2024-09-10T10:07:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f859000f100 con 0x55db8e79f5e0 2024-09-10T10:07:04.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.349+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f859000cef0 con 0x55db8e79ecc0 2024-09-10T10:07:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.350+0000 7f85b19f0640 1 -- 172.21.15.45:0/1025583219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 msgr2=0x55db8e7a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.350+0000 7f85b19f0640 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.350+0000 7f85b19f0640 1 -- 172.21.15.45:0/1025583219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 msgr2=0x55db8e79a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.350+0000 7f85b19f0640 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e79a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.350+0000 7f85b19f0640 1 -- 172.21.15.45:0/1025583219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8e7b6900 con 0x55db8e7a9e80 2024-09-10T10:07:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.351+0000 7f85b19f0640 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7c2240 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85a800d750 tx=0x7f85a800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b773787c9e0b52da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.352+0000 7f8596ffd640 1 -- 172.21.15.45:0/1025583219 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8033040 con 0x55db8e7a9e80 2024-09-10T10:07:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.353+0000 7f8596ffd640 1 -- 172.21.15.45:0/1025583219 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a8021070 con 0x55db8e7a9e80 2024-09-10T10:07:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.355+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 msgr2=0x55db8e7c2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.355+0000 7f85b61ee740 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7c2240 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f85a800d750 tx=0x7f85a800de40 comp rx=0 tx=0).stop 2024-09-10T10:07:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.356+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 shutdown_connections 2024-09-10T10:07:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.356+0000 7f85b61ee740 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7c2240 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.356+0000 7f85b61ee740 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e79a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.356+0000 7f85b61ee740 1 --2- 172.21.15.45:0/1025583219 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.356+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 >> 172.21.15.45:0/1025583219 conn(0x55db8e79b230 msgr2=0x55db8e799a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.357+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 shutdown_connections 2024-09-10T10:07:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.357+0000 7f85b61ee740 1 -- 172.21.15.45:0/1025583219 wait complete. 2024-09-10T10:07:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.358+0000 7f85b61ee740 1 -- start start 2024-09-10T10:07:04.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.359+0000 7f85b61ee740 1 Processor -- start 2024-09-10T10:07:04.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.360+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.361+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.361+0000 7f85b61ee740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.361+0000 7f85b61ee740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f859000d5c0 con 0x55db8e7a9e80 2024-09-10T10:07:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.361+0000 7f85b61ee740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f859000ea30 con 0x55db8e79f5e0 2024-09-10T10:07:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.361+0000 7f85b61ee740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f859000dc90 con 0x55db8e79ecc0 2024-09-10T10:07:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.362+0000 7f85b61ee740 10 client.0 _ll_register_callbacks cb 0x7f85a800d680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.364+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.365+0000 7f85b11ef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:60336/0 (socket says 172.21.15.45:60336) 2024-09-10T10:07:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.365+0000 7f85b11ef640 1 -- 172.21.15.45:0/780606782 learned_addr learned my addr 172.21.15.45:0/780606782 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.365+0000 7f85b09ee640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.365+0000 7f85b19f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.366+0000 7f85b19f0640 1 -- 172.21.15.45:0/780606782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 msgr2=0x55db8e7a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.366+0000 7f85b19f0640 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.366+0000 7f85b19f0640 1 -- 172.21.15.45:0/780606782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 msgr2=0x55db8e7b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.366+0000 7f85b19f0640 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.366+0000 7f85b19f0640 1 -- 172.21.15.45:0/780606782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8e7b2b10 con 0x55db8e7a9e80 2024-09-10T10:07:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.368+0000 7f85b19f0640 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7b23d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85a801c970 tx=0x7f85a801ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a93ef4562341781 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.369+0000 7f8594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:07:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.370+0000 7f8594ff9640 1 -- 172.21.15.45:0/780606782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8044060 con 0x55db8e7a9e80 2024-09-10T10:07:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.370+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a8021070 con 0x55db8e7a9e80 2024-09-10T10:07:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.371+0000 7f8594ff9640 1 -- 172.21.15.45:0/780606782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f85a801c630 con 0x55db8e7a9e80 2024-09-10T10:07:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.372+0000 7f8594ff9640 1 -- 172.21.15.45:0/780606782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f85a8033040 con 0x55db8e7a9e80 2024-09-10T10:07:04.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.372+0000 7f85b61ee740 10 client.15375 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.372+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55db8e824300 con 0x55db8e7a9e80 2024-09-10T10:07:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.374+0000 7f8594ff9640 1 -- 172.21.15.45:0/780606782 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a8021070 con 0x55db8e7a9e80 2024-09-10T10:07:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.377+0000 7f8594ff9640 1 -- 172.21.15.45:0/780606782 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f85a8036070 con 0x55db8e7a9e80 2024-09-10T10:07:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.377+0000 7f85b61ee740 10 client.15375 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.377+0000 7f85b61ee740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.377+0000 7f85b61ee740 20 client.15375 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.377+0000 7f85b61ee740 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e8248c0 0x55db8e826cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.377+0000 7f85b61ee740 4 client.15375 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:07:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.377+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55db8e687900 con 0x55db8e8248c0 2024-09-10T10:07:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.382+0000 7f85b11ef640 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e8248c0 0x55db8e826cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.391+0000 7f85b11ef640 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e8248c0 0x55db8e826cb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af65a90db92970 server_cookie=b877546262af9661 in_seq=0 out_seq=0 2024-09-10T10:07:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:04.392+0000 7f8594ff9640 10 client.15375 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:05 smithi162 ceph-mon[34963]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 174 GiB used, 898 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 993 MiB/s wr, 3.07k op/s 2024-09-10T10:07:05.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:05 smithi045 ceph-mon[31114]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 174 GiB used, 898 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 993 MiB/s wr, 3.07k op/s 2024-09-10T10:07:05.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:05 smithi047 ceph-mon[35244]: pgmap v119: 97 pgs: 97 active+clean; 30 GiB data, 174 GiB used, 898 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 993 MiB/s wr, 3.07k op/s 2024-09-10T10:07:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.491+0000 7f8594ff9640 1 -- 172.21.15.45:0/780606782 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55db8e687900 con 0x55db8e8248c0 2024-09-10T10:07:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.491+0000 7f8594ff9640 10 client.15375 handle_command_reply: tid=0 2024-09-10T10:07:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.491+0000 7f85b61ee740 1 client.15375 shutdown 2024-09-10T10:07:06.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.491+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 msgr2=0x55db8e7b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.491+0000 7f85b61ee740 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7b23d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f85a801c970 tx=0x7f85a801ce10 comp rx=0 tx=0).stop 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 shutdown_connections 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55db8e8248c0 0x55db8e826cb0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55db8e7a9e80 0x55db8e7b23d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55db8e79f5e0 0x55db8e7b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 --2- 172.21.15.45:0/780606782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55db8e79ecc0 0x55db8e7a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 >> 172.21.15.45:0/780606782 conn(0x55db8e79b230 msgr2=0x55db8e79a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f8594ff9640 0 client.15375 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 shutdown_connections 2024-09-10T10:07:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 1 -- 172.21.15.45:0/780606782 wait complete. 2024-09-10T10:07:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.492+0000 7f85b61ee740 20 client.15375 trim_cache size 0 max 16384 2024-09-10T10:07:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:07:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:07:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:07:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:07:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.493+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f85980a2d30 msgr2=0x7f85980a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.493+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f85980a2d30 0x7f85980a51f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85a8005fd0 tx=0x7f85a80046c0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.493+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 msgr2=0x7f85ac113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.493+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 0x7f85ac113ff0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d950 tx=0x7f859c00de10 comp rx=0 tx=0).stop 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 shutdown_connections 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f85980a2d30 0x7f85980a51f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85ac114ab0 0x7f85ac10b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f85ac109b10 0x7f85ac114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 --2- 172.21.15.45:0/3817912786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85ac1091f0 0x7f85ac113ff0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 >> 172.21.15.45:0/3817912786 conn(0x7f85ac06cbf0 msgr2=0x7f85ac071af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 shutdown_connections 2024-09-10T10:07:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:06.494+0000 7f85b21f1640 1 -- 172.21.15.45:0/3817912786 wait complete. 2024-09-10T10:07:07.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:07 smithi162 ceph-mon[34963]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 174 GiB used, 898 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 727 MiB/s wr, 2.47k op/s 2024-09-10T10:07:07.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:07 smithi045 ceph-mon[31114]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 174 GiB used, 898 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 727 MiB/s wr, 2.47k op/s 2024-09-10T10:07:07.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:07 smithi047 ceph-mon[35244]: pgmap v120: 97 pgs: 97 active+clean; 30 GiB data, 174 GiB used, 898 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 727 MiB/s wr, 2.47k op/s 2024-09-10T10:07:09.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:09 smithi045 ceph-mon[31114]: pgmap v121: 97 pgs: 97 active+clean; 31 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 790 MiB/s wr, 2.78k op/s 2024-09-10T10:07:09.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:09 smithi047 ceph-mon[35244]: pgmap v121: 97 pgs: 97 active+clean; 31 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 790 MiB/s wr, 2.78k op/s 2024-09-10T10:07:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:09 smithi162 ceph-mon[34963]: pgmap v121: 97 pgs: 97 active+clean; 31 GiB data, 176 GiB used, 897 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 790 MiB/s wr, 2.78k op/s 2024-09-10T10:07:11.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:10 smithi162 ceph-mon[34963]: pgmap v122: 97 pgs: 97 active+clean; 32 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 934 MiB/s wr, 3.33k op/s 2024-09-10T10:07:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:10 smithi045 ceph-mon[31114]: pgmap v122: 97 pgs: 97 active+clean; 32 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 934 MiB/s wr, 3.33k op/s 2024-09-10T10:07:11.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:10 smithi047 ceph-mon[35244]: pgmap v122: 97 pgs: 97 active+clean; 32 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 934 MiB/s wr, 3.33k op/s 2024-09-10T10:07:13.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:12 smithi162 ceph-mon[34963]: pgmap v123: 97 pgs: 97 active+clean; 32 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 695 MiB/s wr, 2.63k op/s 2024-09-10T10:07:13.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:12 smithi045 ceph-mon[31114]: pgmap v123: 97 pgs: 97 active+clean; 32 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 695 MiB/s wr, 2.63k op/s 2024-09-10T10:07:13.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:12 smithi047 ceph-mon[35244]: pgmap v123: 97 pgs: 97 active+clean; 32 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 695 MiB/s wr, 2.63k op/s 2024-09-10T10:07:15.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:14 smithi162 ceph-mon[34963]: pgmap v124: 97 pgs: 97 active+clean; 34 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 978 MiB/s wr, 3.80k op/s 2024-09-10T10:07:15.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:14 smithi045 ceph-mon[31114]: pgmap v124: 97 pgs: 97 active+clean; 34 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 978 MiB/s wr, 3.80k op/s 2024-09-10T10:07:15.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:14 smithi047 ceph-mon[35244]: pgmap v124: 97 pgs: 97 active+clean; 34 GiB data, 191 GiB used, 882 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 978 MiB/s wr, 3.80k op/s 2024-09-10T10:07:17.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:16 smithi045 ceph-mon[31114]: pgmap v125: 97 pgs: 97 active+clean; 35 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 723 MiB/s wr, 2.72k op/s 2024-09-10T10:07:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:16 smithi162 ceph-mon[34963]: pgmap v125: 97 pgs: 97 active+clean; 35 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 723 MiB/s wr, 2.72k op/s 2024-09-10T10:07:17.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:16 smithi047 ceph-mon[35244]: pgmap v125: 97 pgs: 97 active+clean; 35 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 723 MiB/s wr, 2.72k op/s 2024-09-10T10:07:19.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:19 smithi162 ceph-mon[34963]: pgmap v126: 97 pgs: 97 active+clean; 36 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 781 MiB/s wr, 3.09k op/s 2024-09-10T10:07:19.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:19 smithi045 ceph-mon[31114]: pgmap v126: 97 pgs: 97 active+clean; 36 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 781 MiB/s wr, 3.09k op/s 2024-09-10T10:07:19.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:19 smithi047 ceph-mon[35244]: pgmap v126: 97 pgs: 97 active+clean; 36 GiB data, 195 GiB used, 878 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 781 MiB/s wr, 3.09k op/s 2024-09-10T10:07:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.904+0000 7f91ca12f640 1 -- 172.21.15.45:0/800320104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c410d2d0 msgr2=0x7f91c4114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.904+0000 7f91ca12f640 1 --2- 172.21.15.45:0/800320104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c410d2d0 0x7f91c4114e20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f91b401a910 tx=0x7f91b403b450 comp rx=0 tx=0).stop 2024-09-10T10:07:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.904+0000 7f91ca12f640 1 -- 172.21.15.45:0/800320104 shutdown_connections 2024-09-10T10:07:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.904+0000 7f91ca12f640 1 --2- 172.21.15.45:0/800320104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c410d2d0 0x7f91c4114e20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.904+0000 7f91ca12f640 1 --2- 172.21.15.45:0/800320104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c4108c40 0x7f91c410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.904+0000 7f91ca12f640 1 --2- 172.21.15.45:0/800320104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4108290 0x7f91c4108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.904+0000 7f91ca12f640 1 -- 172.21.15.45:0/800320104 >> 172.21.15.45:0/800320104 conn(0x7f91c40778f0 msgr2=0x7f91c4079d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.914+0000 7f91ca12f640 1 -- 172.21.15.45:0/800320104 shutdown_connections 2024-09-10T10:07:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.915+0000 7f91ca12f640 1 -- 172.21.15.45:0/800320104 wait complete. 2024-09-10T10:07:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.916+0000 7f91ca12f640 1 Processor -- start 2024-09-10T10:07:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.916+0000 7f91ca12f640 1 -- start start 2024-09-10T10:07:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.916+0000 7f91ca12f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c4108290 0x7f91c4114a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.916+0000 7f91ca12f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c410d2d0 0x7f91c4110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.916+0000 7f91ca12f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4111050 0x7f91c4111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91ca12f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91c406e5e0 con 0x7f91c4108290 2024-09-10T10:07:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91ca12f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f91c406e460 con 0x7f91c410d2d0 2024-09-10T10:07:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91ca12f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f91c406e760 con 0x7f91c4111050 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4111050 0x7f91c4111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4111050 0x7f91c4111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:32832/0 (socket says 172.21.15.45:32832) 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 -- 172.21.15.45:0/1616722855 learned_addr learned my addr 172.21.15.45:0/1616722855 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 -- 172.21.15.45:0/1616722855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c410d2d0 msgr2=0x7f91c4110b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c410d2d0 0x7f91c4110b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 -- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c4108290 msgr2=0x7f91c4114a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c4108290 0x7f91c4114a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.917+0000 7f91c992e640 1 -- 172.21.15.45:0/1616722855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c41c4710 con 0x7f91c4111050 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.918+0000 7f91c992e640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4111050 0x7f91c4111500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91b403b9c0 tx=0x7f91b4004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.919+0000 7f91ba7fc640 1 -- 172.21.15.45:0/1616722855 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4012020 con 0x7f91c4111050 2024-09-10T10:07:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.919+0000 7f91ca12f640 1 -- 172.21.15.45:0/1616722855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c41c49a0 con 0x7f91c4111050 2024-09-10T10:07:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.919+0000 7f91ca12f640 1 -- 172.21.15.45:0/1616722855 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c41c4da0 con 0x7f91c4111050 2024-09-10T10:07:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.928+0000 7f91ce20a740 1 Processor -- start 2024-09-10T10:07:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.928+0000 7f91ce20a740 1 -- start start 2024-09-10T10:07:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.928+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b29f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.928+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.928+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2aa1c0 0x563e3b2ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.928+0000 7f91ce20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563e3b2032f0 con 0x563e3b2aa1c0 2024-09-10T10:07:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.929+0000 7f91ce20a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563e3b1ab270 con 0x563e3b29f920 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.929+0000 7f91ce20a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563e3b1ff2b0 con 0x563e3b29f000 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.929+0000 7f91c912d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.929+0000 7f91c912d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:40178/0 (socket says 172.21.15.45:40178) 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.929+0000 7f91c912d640 1 -- 172.21.15.45:0/623445869 learned_addr learned my addr 172.21.15.45:0/623445869 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.929+0000 7f91c892c640 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b29f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.930+0000 7f91c892c640 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2aa1c0 0x563e3b2ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.930+0000 7f91ba7fc640 1 -- 172.21.15.45:0/1616722855 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b4040c00 con 0x7f91c4111050 2024-09-10T10:07:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.930+0000 7f91ba7fc640 1 -- 172.21.15.45:0/1616722855 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4046030 con 0x7f91c4111050 2024-09-10T10:07:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.949+0000 7f91ba7fc640 1 -- 172.21.15.45:0/1616722855 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f91b4015020 con 0x7f91c4111050 2024-09-10T10:07:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.950+0000 7f91ba7fc640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f91b00a2f00 0x7f91b00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.951+0000 7f91c992e640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f91b00a2f00 0x7f91b00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.951+0000 7f91ba7fc640 1 -- 172.21.15.45:0/1616722855 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f91b40f06c0 con 0x7f91c4111050 2024-09-10T10:07:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.952+0000 7f91c892c640 1 -- 172.21.15.45:0/623445869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 msgr2=0x563e3b29f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.952+0000 7f91c892c640 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b29f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.952+0000 7f91c892c640 1 -- 172.21.15.45:0/623445869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 msgr2=0x563e3b2a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.952+0000 7f91c892c640 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.952+0000 7f91c892c640 1 -- 172.21.15.45:0/623445869 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3b1aa340 con 0x563e3b2aa1c0 2024-09-10T10:07:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.952+0000 7f91c992e640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f91b00a2f00 0x7f91b00a53c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f91b4005ab0 tx=0x7f91b4040db0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.954+0000 7f91c912d640 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:07:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.962+0000 7f91c892c640 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2aa1c0 0x563e3b2ac5b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f91c41122d0 tx=0x7f91c00068d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357b21de77ed5ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.970+0000 7f91aeffd640 1 -- 172.21.15.45:0/623445869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0011040 con 0x563e3b2aa1c0 2024-09-10T10:07:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.970+0000 7f91aeffd640 1 -- 172.21.15.45:0/623445869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c000bba0 con 0x563e3b2aa1c0 2024-09-10T10:07:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.971+0000 7f91ce20a740 1 -- 172.21.15.45:0/623445869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2aa1c0 msgr2=0x563e3b2ac5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91aeffd640 1 -- 172.21.15.45:0/623445869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0011040 con 0x563e3b2aa1c0 2024-09-10T10:07:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2aa1c0 0x563e3b2ac5b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f91c41122d0 tx=0x7f91c00068d0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 -- 172.21.15.45:0/623445869 shutdown_connections 2024-09-10T10:07:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2aa1c0 0x563e3b2ac5b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 --2- 172.21.15.45:0/623445869 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b29f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 -- 172.21.15.45:0/623445869 >> 172.21.15.45:0/623445869 conn(0x563e3b29b570 msgr2=0x563e3b29b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 -- 172.21.15.45:0/623445869 shutdown_connections 2024-09-10T10:07:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.972+0000 7f91ce20a740 1 -- 172.21.15.45:0/623445869 wait complete. 2024-09-10T10:07:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.973+0000 7f91ce20a740 1 -- start start 2024-09-10T10:07:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.973+0000 7f91ce20a740 1 Processor -- start 2024-09-10T10:07:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.974+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f000 0x563e3b2a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.974+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.975+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b2a7a40 0x563e3b2b6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.975+0000 7f91ce20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563e3b2acd00 con 0x563e3b29f000 2024-09-10T10:07:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.975+0000 7f91ce20a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563e3b18c290 con 0x563e3b29f920 2024-09-10T10:07:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.975+0000 7f91ce20a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x563e3b2ace80 con 0x563e3b2a7a40 2024-09-10T10:07:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.975+0000 7f91ce20a740 10 client.0 _ll_register_callbacks cb 0x563e3b1bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.980+0000 7f91c912d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.980+0000 7f91c912d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:40182/0 (socket says 172.21.15.45:40182) 2024-09-10T10:07:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.980+0000 7f91c912d640 1 -- 172.21.15.45:0/2064380670 learned_addr learned my addr 172.21.15.45:0/2064380670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.983+0000 7f91c892c640 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f000 0x563e3b2a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.983+0000 7f91c892c640 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b2a7a40 0x563e3b2b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.986+0000 7f91c912d640 1 -- 172.21.15.45:0/2064380670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b2a7a40 msgr2=0x563e3b2b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.986+0000 7f91c912d640 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b2a7a40 0x563e3b2b6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.987+0000 7f91c912d640 1 -- 172.21.15.45:0/2064380670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f000 msgr2=0x563e3b2a6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.987+0000 7f91c912d640 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f000 0x563e3b2a6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.987+0000 7f91c912d640 1 -- 172.21.15.45:0/2064380670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3af16210 con 0x563e3b29f920 2024-09-10T10:07:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.990+0000 7f91c912d640 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a7500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009e30 tx=0x7f91bc004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b2b9bc8eb8f633 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.990+0000 7f91acff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:07:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.991+0000 7f91acff9640 1 -- 172.21.15.45:0/2064380670 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc016070 con 0x563e3b29f920 2024-09-10T10:07:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.991+0000 7f91acff9640 1 -- 172.21.15.45:0/2064380670 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f91bc012920 con 0x563e3b29f920 2024-09-10T10:07:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.991+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c000bba0 con 0x563e3b29f920 2024-09-10T10:07:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.994+0000 7f91acff9640 1 -- 172.21.15.45:0/2064380670 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f91bc012b20 con 0x563e3b29f920 2024-09-10T10:07:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.994+0000 7f91ce20a740 10 client.25079 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.994+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563e3afe2620 con 0x563e3b29f920 2024-09-10T10:07:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91acff9640 1 -- 172.21.15.45:0/2064380670 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c000bba0 con 0x563e3b29f920 2024-09-10T10:07:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91acff9640 1 -- 172.21.15.45:0/2064380670 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f91bc00f040 con 0x563e3b29f920 2024-09-10T10:07:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91ce20a740 10 client.25079 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91ce20a740 10 client.25079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91ce20a740 20 client.25079 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b2ba3c0 0x563e3b2bc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91ce20a740 4 client.25079 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:07:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.995+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e3b210a20 con 0x563e3b2ba3c0 2024-09-10T10:07:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:19.997+0000 7f91c892c640 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b2ba3c0 0x563e3b2bc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.010+0000 7f91acff9640 10 client.25079 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.010+0000 7f91c892c640 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b2ba3c0 0x563e3b2bc7b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4f781a777011fe server_cookie=7dd3ac84b079b6c2 in_seq=0 out_seq=0 2024-09-10T10:07:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.014+0000 7f91acff9640 1 -- 172.21.15.45:0/2064380670 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e3b210a20 con 0x563e3b2ba3c0 2024-09-10T10:07:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.014+0000 7f91acff9640 10 client.25079 handle_command_reply: tid=0 2024-09-10T10:07:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.014+0000 7f91ce20a740 1 client.25079 shutdown 2024-09-10T10:07:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.014+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 msgr2=0x563e3b2a7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.014+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a7500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009e30 tx=0x7f91bc004410 comp rx=0 tx=0).stop 2024-09-10T10:07:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.017+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 shutdown_connections 2024-09-10T10:07:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.017+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b2ba3c0 0x563e3b2bc7b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.017+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b2a7a40 0x563e3b2b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.017+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f920 0x563e3b2a7500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.017+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2064380670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f000 0x563e3b2a6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.017+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 >> 172.21.15.45:0/2064380670 conn(0x563e3b29b570 msgr2=0x563e3b2a1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.017+0000 7f91acff9640 0 client.25079 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.018+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 shutdown_connections 2024-09-10T10:07:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.018+0000 7f91ce20a740 1 -- 172.21.15.45:0/2064380670 wait complete. 2024-09-10T10:07:20.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.018+0000 7f91ce20a740 20 client.25079 trim_cache size 0 max 16384 2024-09-10T10:07:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.059+0000 7f91ce20a740 1 Processor -- start 2024-09-10T10:07:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.059+0000 7f91ce20a740 1 -- start start 2024-09-10T10:07:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.060+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b2b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.060+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f920 0x563e3b2c2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:20.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.060+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b2a7a40 0x563e3b2c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.060+0000 7f91ce20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41cb540 con 0x563e3b29f920 2024-09-10T10:07:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.060+0000 7f91ce20a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41cdde0 con 0x563e3b2a7a40 2024-09-10T10:07:20.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.061+0000 7f91ce20a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41cbbd0 con 0x563e3b29f000 2024-09-10T10:07:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.061+0000 7f91c892c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b2a7a40 0x563e3b2c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.061+0000 7f91c892c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b2a7a40 0x563e3b2c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:40188/0 (socket says 172.21.15.45:40188) 2024-09-10T10:07:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.061+0000 7f91c892c640 1 -- 172.21.15.45:0/2097855357 learned_addr learned my addr 172.21.15.45:0/2097855357 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:20.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.061+0000 7f91c892c640 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b2b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.065+0000 7f91c892c640 1 -- 172.21.15.45:0/2097855357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b2a7a40 msgr2=0x563e3b2c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.065+0000 7f91c892c640 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b2a7a40 0x563e3b2c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.065+0000 7f91c892c640 1 -- 172.21.15.45:0/2097855357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f920 msgr2=0x563e3b2c2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.065+0000 7f91c892c640 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f920 0x563e3b2c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.065+0000 7f91c892c640 1 -- 172.21.15.45:0/2097855357 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3b2a7870 con 0x563e3b29f000 2024-09-10T10:07:20.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.071+0000 7f91c892c640 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b2b5ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91c000b980 tx=0x7f91c000a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80dc0726d7451459 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.079+0000 7f91ae7fc640 1 -- 172.21.15.45:0/2097855357 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c000b6d0 con 0x563e3b29f000 2024-09-10T10:07:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.079+0000 7f91ae7fc640 1 -- 172.21.15.45:0/2097855357 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c0003780 con 0x563e3b29f000 2024-09-10T10:07:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.079+0000 7f91ae7fc640 1 -- 172.21.15.45:0/2097855357 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0018700 con 0x563e3b29f000 2024-09-10T10:07:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.081+0000 7f91ce20a740 1 -- 172.21.15.45:0/2097855357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 msgr2=0x563e3b2b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.081+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b2b5ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91c000b980 tx=0x7f91c000a040 comp rx=0 tx=0).stop 2024-09-10T10:07:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.082+0000 7f91ce20a740 1 -- 172.21.15.45:0/2097855357 shutdown_connections 2024-09-10T10:07:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.082+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b2a7a40 0x563e3b2c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.082+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b29f920 0x563e3b2c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.082+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2097855357 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f000 0x563e3b2b5ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.082+0000 7f91ce20a740 1 -- 172.21.15.45:0/2097855357 >> 172.21.15.45:0/2097855357 conn(0x563e3b29b570 msgr2=0x563e3b2a9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.086+0000 7f91ce20a740 1 -- 172.21.15.45:0/2097855357 shutdown_connections 2024-09-10T10:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.089+0000 7f91ce20a740 1 -- 172.21.15.45:0/2097855357 wait complete. 2024-09-10T10:07:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.090+0000 7f91ce20a740 1 -- start start 2024-09-10T10:07:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.090+0000 7f91ce20a740 1 Processor -- start 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f000 0x563e3b2addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f920 0x563e3b2ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91ce20a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2a7a40 0x563e3b2ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91ce20a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41cc2a0 con 0x563e3b2a7a40 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91ce20a740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41cd710 con 0x563e3b29f000 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91ce20a740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f91c41cc970 con 0x563e3b29f920 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91ce20a740 10 client.0 _ll_register_callbacks cb 0x7f91bc003a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91c912d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f920 0x563e3b2ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91c912d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f920 0x563e3b2ae300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:32874/0 (socket says 172.21.15.45:32874) 2024-09-10T10:07:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.092+0000 7f91c912d640 1 -- 172.21.15.45:0/2933649376 learned_addr learned my addr 172.21.15.45:0/2933649376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.093+0000 7f91c892c640 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f000 0x563e3b2addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.093+0000 7f91c892c640 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2a7a40 0x563e3b2ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.093+0000 7f91c912d640 1 -- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f000 msgr2=0x563e3b2addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.093+0000 7f91c912d640 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f000 0x563e3b2addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.093+0000 7f91c912d640 1 -- 172.21.15.45:0/2933649376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2a7a40 msgr2=0x563e3b2ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.093+0000 7f91c912d640 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2a7a40 0x563e3b2ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.093+0000 7f91c912d640 1 -- 172.21.15.45:0/2933649376 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3b335010 con 0x563e3b29f920 2024-09-10T10:07:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.094+0000 7f91c912d640 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f920 0x563e3b2ae300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f91bc002230 tx=0x7f91bc005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e081b710462fd30c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.094+0000 7f91acff9640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:07:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.094+0000 7f91acff9640 1 -- 172.21.15.45:0/2933649376 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc016040 con 0x563e3b29f920 2024-09-10T10:07:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.094+0000 7f91acff9640 1 -- 172.21.15.45:0/2933649376 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f91bc012920 con 0x563e3b29f920 2024-09-10T10:07:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.094+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c0003780 con 0x563e3b29f920 2024-09-10T10:07:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.095+0000 7f91acff9640 1 -- 172.21.15.45:0/2933649376 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f91bc025a50 con 0x563e3b29f920 2024-09-10T10:07:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.095+0000 7f91ce20a740 10 client.25003 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.095+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563e3b336270 con 0x563e3b29f920 2024-09-10T10:07:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.095+0000 7f91acff9640 1 -- 172.21.15.45:0/2933649376 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c0003780 con 0x563e3b29f920 2024-09-10T10:07:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91acff9640 1 -- 172.21.15.45:0/2933649376 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f91bc004910 con 0x563e3b29f920 2024-09-10T10:07:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91ce20a740 10 client.25003 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91ce20a740 10 client.25003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91ce20a740 20 client.25003 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b336940 0x563e3b338d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91ce20a740 4 client.25003 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:07:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e3b20ad70 con 0x563e3b336940 2024-09-10T10:07:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.097+0000 7f91c892c640 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b336940 0x563e3b338d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.099+0000 7f91c892c640 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b336940 0x563e3b338d60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293264f0b238ec93 server_cookie=e44818c56187b43d in_seq=0 out_seq=0 2024-09-10T10:07:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:20.099+0000 7f91acff9640 10 client.25003 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:21.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:21 smithi162 ceph-mon[34963]: pgmap v127: 97 pgs: 97 active+clean; 37 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 859 MiB/s wr, 3.57k op/s 2024-09-10T10:07:21.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:21 smithi045 ceph-mon[31114]: pgmap v127: 97 pgs: 97 active+clean; 37 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 859 MiB/s wr, 3.57k op/s 2024-09-10T10:07:21.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:21 smithi047 ceph-mon[35244]: pgmap v127: 97 pgs: 97 active+clean; 37 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 859 MiB/s wr, 3.57k op/s 2024-09-10T10:07:22.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:22 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:07:22.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:22 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:07:22.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:22 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:07:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:23 smithi162 ceph-mon[34963]: pgmap v128: 97 pgs: 97 active+clean; 38 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 657 MiB/s wr, 2.95k op/s 2024-09-10T10:07:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:23 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:07:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:23 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:07:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:23 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:07:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:23 smithi045 ceph-mon[31114]: pgmap v128: 97 pgs: 97 active+clean; 38 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 657 MiB/s wr, 2.95k op/s 2024-09-10T10:07:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:23 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:07:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:23 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:07:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:23 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:07:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:23 smithi047 ceph-mon[35244]: pgmap v128: 97 pgs: 97 active+clean; 38 GiB data, 193 GiB used, 879 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 657 MiB/s wr, 2.95k op/s 2024-09-10T10:07:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:23 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:07:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:23 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:07:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:23 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:07:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.486+0000 7f91acff9640 1 -- 172.21.15.45:0/2933649376 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e3b20ad70 con 0x563e3b336940 2024-09-10T10:07:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.486+0000 7f91acff9640 10 client.25003 handle_command_reply: tid=0 2024-09-10T10:07:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.486+0000 7f91ce20a740 1 client.25003 shutdown 2024-09-10T10:07:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.486+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f920 msgr2=0x563e3b2ae300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.486+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f920 0x563e3b2ae300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f91bc002230 tx=0x7f91bc005a60 comp rx=0 tx=0).stop 2024-09-10T10:07:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.487+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 shutdown_connections 2024-09-10T10:07:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.487+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x563e3b336940 0x563e3b338d60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.487+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563e3b2a7a40 0x563e3b2ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.487+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x563e3b29f920 0x563e3b2ae300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.487+0000 7f91ce20a740 1 --2- 172.21.15.45:0/2933649376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563e3b29f000 0x563e3b2addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.487+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 >> 172.21.15.45:0/2933649376 conn(0x563e3b29b570 msgr2=0x563e3b299eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.487+0000 7f91acff9640 0 client.25003 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.489+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 shutdown_connections 2024-09-10T10:07:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.489+0000 7f91ce20a740 1 -- 172.21.15.45:0/2933649376 wait complete. 2024-09-10T10:07:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.490+0000 7f91ce20a740 20 client.25003 trim_cache size 0 max 16384 2024-09-10T10:07:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:07:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:07:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:07:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:07:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.497+0000 7f91affff640 1 -- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f91b00a2f00 msgr2=0x7f91b00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:24.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.497+0000 7f91affff640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f91b00a2f00 0x7f91b00a53c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f91b4005ab0 tx=0x7f91b4040db0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.497+0000 7f91affff640 1 -- 172.21.15.45:0/1616722855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4111050 msgr2=0x7f91c4111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.498+0000 7f91affff640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4111050 0x7f91c4111500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f91b403b9c0 tx=0x7f91b4004cb0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.501+0000 7f91affff640 1 -- 172.21.15.45:0/1616722855 shutdown_connections 2024-09-10T10:07:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.501+0000 7f91affff640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f91c4111050 0x7f91c4111500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.501+0000 7f91affff640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f91b00a2f00 0x7f91b00a53c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.501+0000 7f91affff640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91c410d2d0 0x7f91c4110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.501+0000 7f91affff640 1 --2- 172.21.15.45:0/1616722855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c4108290 0x7f91c4114a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.501+0000 7f91affff640 1 -- 172.21.15.45:0/1616722855 >> 172.21.15.45:0/1616722855 conn(0x7f91c40778f0 msgr2=0x7f91c4078ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.508+0000 7f91affff640 1 -- 172.21.15.45:0/1616722855 shutdown_connections 2024-09-10T10:07:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:24.508+0000 7f91affff640 1 -- 172.21.15.45:0/1616722855 wait complete. 2024-09-10T10:07:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:25 smithi162 ceph-mon[34963]: pgmap v129: 97 pgs: 97 active+clean; 38 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 956 MiB/s wr, 3.66k op/s 2024-09-10T10:07:25.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:25 smithi045 ceph-mon[31114]: pgmap v129: 97 pgs: 97 active+clean; 38 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 956 MiB/s wr, 3.66k op/s 2024-09-10T10:07:25.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:25 smithi047 ceph-mon[35244]: pgmap v129: 97 pgs: 97 active+clean; 38 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 956 MiB/s wr, 3.66k op/s 2024-09-10T10:07:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:27 smithi162 ceph-mon[34963]: pgmap v130: 97 pgs: 97 active+clean; 38 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 715 MiB/s wr, 2.52k op/s 2024-09-10T10:07:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:27 smithi045 ceph-mon[31114]: pgmap v130: 97 pgs: 97 active+clean; 38 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 715 MiB/s wr, 2.52k op/s 2024-09-10T10:07:27.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:27 smithi047 ceph-mon[35244]: pgmap v130: 97 pgs: 97 active+clean; 38 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 715 MiB/s wr, 2.52k op/s 2024-09-10T10:07:29.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:29 smithi045 ceph-mon[31114]: pgmap v131: 97 pgs: 97 active+clean; 39 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 786 MiB/s wr, 2.47k op/s 2024-09-10T10:07:29.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:29 smithi047 ceph-mon[35244]: pgmap v131: 97 pgs: 97 active+clean; 39 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 786 MiB/s wr, 2.47k op/s 2024-09-10T10:07:29.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:29 smithi162 ceph-mon[34963]: pgmap v131: 97 pgs: 97 active+clean; 39 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 786 MiB/s wr, 2.47k op/s 2024-09-10T10:07:31.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:31 smithi162 ceph-mon[34963]: pgmap v132: 97 pgs: 97 active+clean; 39 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 890 MiB/s wr, 2.42k op/s 2024-09-10T10:07:31.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:31 smithi045 ceph-mon[31114]: pgmap v132: 97 pgs: 97 active+clean; 39 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 890 MiB/s wr, 2.42k op/s 2024-09-10T10:07:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:31 smithi047 ceph-mon[35244]: pgmap v132: 97 pgs: 97 active+clean; 39 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 890 MiB/s wr, 2.42k op/s 2024-09-10T10:07:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:33 smithi162 ceph-mon[34963]: pgmap v133: 97 pgs: 97 active+clean; 40 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 751 MiB/s wr, 1.71k op/s 2024-09-10T10:07:33.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:33 smithi045 ceph-mon[31114]: pgmap v133: 97 pgs: 97 active+clean; 40 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 751 MiB/s wr, 1.71k op/s 2024-09-10T10:07:33.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:33 smithi047 ceph-mon[35244]: pgmap v133: 97 pgs: 97 active+clean; 40 GiB data, 198 GiB used, 875 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 751 MiB/s wr, 1.71k op/s 2024-09-10T10:07:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:35 smithi162 ceph-mon[34963]: pgmap v134: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 979 MiB/s wr, 2.33k op/s 2024-09-10T10:07:35.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:35 smithi045 ceph-mon[31114]: pgmap v134: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 979 MiB/s wr, 2.33k op/s 2024-09-10T10:07:35.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:35 smithi047 ceph-mon[35244]: pgmap v134: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 979 MiB/s wr, 2.33k op/s 2024-09-10T10:07:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.792+0000 7f199cf3b640 1 -- 172.21.15.45:0/1127723964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1998109b10 msgr2=0x7f1998114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.792+0000 7f199cf3b640 1 --2- 172.21.15.45:0/1127723964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1998109b10 0x7f1998114570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f198c009900 tx=0x7f198c02a400 comp rx=0 tx=0).stop 2024-09-10T10:07:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.792+0000 7f199cf3b640 1 -- 172.21.15.45:0/1127723964 shutdown_connections 2024-09-10T10:07:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.792+0000 7f199cf3b640 1 --2- 172.21.15.45:0/1127723964 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1998114ab0 0x7f1998116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.792+0000 7f199cf3b640 1 --2- 172.21.15.45:0/1127723964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1998109b10 0x7f1998114570 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f198c009900 tx=0x7f198c02a400 comp rx=0 tx=0).stop 2024-09-10T10:07:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.792+0000 7f199cf3b640 1 --2- 172.21.15.45:0/1127723964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 0x7f19981095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.792+0000 7f199cf3b640 1 -- 172.21.15.45:0/1127723964 >> 172.21.15.45:0/1127723964 conn(0x7f199806cc60 msgr2=0x7f199806d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.793+0000 7f199cf3b640 1 -- 172.21.15.45:0/1127723964 shutdown_connections 2024-09-10T10:07:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.793+0000 7f199cf3b640 1 -- 172.21.15.45:0/1127723964 wait complete. 2024-09-10T10:07:37.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 Processor -- start 2024-09-10T10:07:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 -- start start 2024-09-10T10:07:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 0x7f1998112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1998114ab0 0x7f1998112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19981135d0 0x7f19981b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1998118300 con 0x7f19981091f0 2024-09-10T10:07:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1998118180 con 0x7f1998114ab0 2024-09-10T10:07:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.795+0000 7f199cf3b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1998118480 con 0x7f19981135d0 2024-09-10T10:07:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.796+0000 7f199659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 0x7f1998112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.796+0000 7f199659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 0x7f1998112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50424/0 (socket says 172.21.15.45:50424) 2024-09-10T10:07:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.796+0000 7f199659c640 1 -- 172.21.15.45:0/2882805387 learned_addr learned my addr 172.21.15.45:0/2882805387 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.796+0000 7f1996d9d640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19981135d0 0x7f19981b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.797+0000 7f199659c640 1 -- 172.21.15.45:0/2882805387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19981135d0 msgr2=0x7f19981b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.797+0000 7f199659c640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19981135d0 0x7f19981b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.797+0000 7f199659c640 1 -- 172.21.15.45:0/2882805387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1998114ab0 msgr2=0x7f1998112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.797+0000 7f199659c640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1998114ab0 0x7f1998112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.797+0000 7f199659c640 1 -- 172.21.15.45:0/2882805387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19981b38c0 con 0x7f19981091f0 2024-09-10T10:07:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.797+0000 7f199659c640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 0x7f1998112ab0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f199000ebc0 tx=0x7f199000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.798+0000 7f19877fe640 1 -- 172.21.15.45:0/2882805387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990010630 con 0x7f19981091f0 2024-09-10T10:07:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.798+0000 7f19877fe640 1 -- 172.21.15.45:0/2882805387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1990010cb0 con 0x7f19981091f0 2024-09-10T10:07:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.798+0000 7f19877fe640 1 -- 172.21.15.45:0/2882805387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990018bd0 con 0x7f19981091f0 2024-09-10T10:07:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.798+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19981b3bb0 con 0x7f19981091f0 2024-09-10T10:07:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.798+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19981b40b0 con 0x7f19981091f0 2024-09-10T10:07:37.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.803+0000 7f199fcd6740 1 Processor -- start 2024-09-10T10:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.803+0000 7f199fcd6740 1 -- start start 2024-09-10T10:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.803+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc4450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.804+0000 7f19877fe640 1 -- 172.21.15.45:0/2882805387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f19900107d0 con 0x7f19981091f0 2024-09-10T10:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.804+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc44f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.804+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc44fe80 0x5569bc452270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.804+0000 7f199fcd6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5569bc3a92f0 con 0x5569bc4455e0 2024-09-10T10:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.804+0000 7f199fcd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5569bc351270 con 0x5569bc44fe80 2024-09-10T10:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.804+0000 7f199fcd6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5569bc3a52b0 con 0x5569bc444cc0 2024-09-10T10:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.805+0000 7f1996d9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc4450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.805+0000 7f199659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc44f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.805+0000 7f1996d9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc4450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:35474/0 (socket says 172.21.15.45:35474) 2024-09-10T10:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.805+0000 7f1996d9d640 1 -- 172.21.15.45:0/338195800 learned_addr learned my addr 172.21.15.45:0/338195800 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.805+0000 7f1995d9b640 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc44fe80 0x5569bc452270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.806+0000 7f19877fe640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f19800a2da0 0x7f19800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.806+0000 7f1996d9d640 1 -- 172.21.15.45:0/338195800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc44fe80 msgr2=0x5569bc452270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.806+0000 7f1996d9d640 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc44fe80 0x5569bc452270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.806+0000 7f1996d9d640 1 -- 172.21.15.45:0/338195800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 msgr2=0x5569bc44f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.806+0000 7f1996d9d640 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc44f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.806+0000 7f1996d9d640 1 -- 172.21.15.45:0/338195800 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569bc350340 con 0x5569bc444cc0 2024-09-10T10:07:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.807+0000 7f19877fe640 1 -- 172.21.15.45:0/2882805387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f19900cfa60 con 0x7f19981091f0 2024-09-10T10:07:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.807+0000 7f1996d9d640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f19800a2da0 0x7f19800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.808+0000 7f1996d9d640 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc4450a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1988009d20 tx=0x7f1988011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a76249b7ae37a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.809+0000 7f1996d9d640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f19800a2da0 0x7f19800a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f198801c000 tx=0x7f198801e000 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.809+0000 7f1984ff9640 1 -- 172.21.15.45:0/338195800 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988017070 con 0x5569bc444cc0 2024-09-10T10:07:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.809+0000 7f1984ff9640 1 -- 172.21.15.45:0/338195800 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1988004c80 con 0x5569bc444cc0 2024-09-10T10:07:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.809+0000 7f1984ff9640 1 -- 172.21.15.45:0/338195800 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198801d410 con 0x5569bc444cc0 2024-09-10T10:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.810+0000 7f199fcd6740 1 -- 172.21.15.45:0/338195800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 msgr2=0x5569bc4450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.810+0000 7f199fcd6740 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc4450a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1988009d20 tx=0x7f1988011810 comp rx=0 tx=0).stop 2024-09-10T10:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.811+0000 7f199fcd6740 1 -- 172.21.15.45:0/338195800 shutdown_connections 2024-09-10T10:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.811+0000 7f199fcd6740 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc44fe80 0x5569bc452270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.811+0000 7f199fcd6740 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc44f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.811+0000 7f199fcd6740 1 --2- 172.21.15.45:0/338195800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc4450a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.811+0000 7f199fcd6740 1 -- 172.21.15.45:0/338195800 >> 172.21.15.45:0/338195800 conn(0x5569bc441230 msgr2=0x5569bc441610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.812+0000 7f199fcd6740 1 -- 172.21.15.45:0/338195800 shutdown_connections 2024-09-10T10:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.812+0000 7f199fcd6740 1 -- 172.21.15.45:0/338195800 wait complete. 2024-09-10T10:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.812+0000 7f199fcd6740 1 -- start start 2024-09-10T10:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.812+0000 7f199fcd6740 1 Processor -- start 2024-09-10T10:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.813+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc44cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.813+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc4455e0 0x5569bc44d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.813+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.813+0000 7f199659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc44cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.813+0000 7f199659c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc44cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:35486/0 (socket says 172.21.15.45:35486) 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.813+0000 7f199659c640 1 -- 172.21.15.45:0/3438417313 learned_addr learned my addr 172.21.15.45:0/3438417313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.813+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5569bc452aa0 con 0x5569bc44fe80 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5569bc332290 con 0x5569bc4455e0 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5569bc452c20 con 0x5569bc444cc0 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f1996d9d640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f1995d9b640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc4455e0 0x5569bc44d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199659c640 1 -- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc4455e0 msgr2=0x5569bc44d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199659c640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc4455e0 0x5569bc44d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199fcd6740 10 client.0 _ll_register_callbacks cb 0x5569bc363140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199659c640 1 -- 172.21.15.45:0/3438417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 msgr2=0x5569bc44d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199659c640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199659c640 1 -- 172.21.15.45:0/3438417313 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569bc0bc210 con 0x5569bc444cc0 2024-09-10T10:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1988004c80 con 0x5569bc444cc0 2024-09-10T10:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.814+0000 7f1996d9d640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.815+0000 7f199659c640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc44cd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f199000cba0 tx=0x7f1990009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed00f5837fed07c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.815+0000 7f197dffb640 10 client.25015 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.815+0000 7f197dffb640 1 -- 172.21.15.45:0/3438417313 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199009bd60 con 0x5569bc444cc0 2024-09-10T10:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.815+0000 7f197dffb640 1 -- 172.21.15.45:0/3438417313 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f19900babd0 con 0x5569bc444cc0 2024-09-10T10:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.815+0000 7f197dffb640 1 -- 172.21.15.45:0/3438417313 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f199009b8f0 con 0x5569bc444cc0 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.815+0000 7f199fcd6740 10 client.25015 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.815+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({fsmap=19}) -- 0x5569bc188620 con 0x5569bc444cc0 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.816+0000 7f197dffb640 1 -- 172.21.15.45:0/3438417313 <== mon.1 v2:172.21.15.162:3300/0 4 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f199009b3e0 con 0x5569bc444cc0 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.816+0000 7f199fcd6740 10 client.25015 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.816+0000 7f199fcd6740 10 client.25015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.816+0000 7f199fcd6740 20 client.25015 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.816+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.816+0000 7f199fcd6740 4 client.25015 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.816+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569bc3b6a20 con 0x5569bc45fd00 2024-09-10T10:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.817+0000 7f1995d9b640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.818+0000 7f197dffb640 10 client.25015 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.818+0000 7f1995d9b640 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4620f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e615ebe9ba6e3b server_cookie=b8b6d05c717a5ae6 in_seq=0 out_seq=0 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.819+0000 7f197dffb640 1 -- 172.21.15.45:0/3438417313 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569bc3b6a20 con 0x5569bc45fd00 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.819+0000 7f197dffb640 10 client.25015 handle_command_reply: tid=0 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 client.25015 shutdown 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 msgr2=0x5569bc44cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc44cd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f199000cba0 tx=0x7f1990009200 comp rx=0 tx=0).stop 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 shutdown_connections 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4620f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f197dffb640 0 client.25015 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc4455e0 0x5569bc44d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3438417313 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc444cc0 0x5569bc44cd80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 >> 172.21.15.45:0/3438417313 conn(0x5569bc441230 msgr2=0x5569bc446f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.820+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 shutdown_connections 2024-09-10T10:07:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.821+0000 7f199fcd6740 1 -- 172.21.15.45:0/3438417313 wait complete. 2024-09-10T10:07:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.821+0000 7f199fcd6740 20 client.25015 trim_cache size 0 max 16384 2024-09-10T10:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.855+0000 7f199fcd6740 1 Processor -- start 2024-09-10T10:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.855+0000 7f199fcd6740 1 -- start start 2024-09-10T10:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.855+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 0x5569bc447780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.856+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc4455e0 0x5569bc440170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.856+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.856+0000 7f199fcd6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19981baa90 con 0x5569bc44fe80 2024-09-10T10:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.856+0000 7f199fcd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f19981bd330 con 0x5569bc444cc0 2024-09-10T10:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.856+0000 7f199fcd6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f19981bb120 con 0x5569bc4455e0 2024-09-10T10:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.857+0000 7f1995d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc4455e0 0x5569bc440170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.857+0000 7f1995d9b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc4455e0 0x5569bc440170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:35502/0 (socket says 172.21.15.45:35502) 2024-09-10T10:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.857+0000 7f1995d9b640 1 -- 172.21.15.45:0/732524200 learned_addr learned my addr 172.21.15.45:0/732524200 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.857+0000 7f1996d9d640 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.857+0000 7f199659c640 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 0x5569bc447780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.858+0000 7f1996d9d640 1 -- 172.21.15.45:0/732524200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc4455e0 msgr2=0x5569bc440170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.858+0000 7f1996d9d640 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc4455e0 0x5569bc440170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.858+0000 7f1996d9d640 1 -- 172.21.15.45:0/732524200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 msgr2=0x5569bc447780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.858+0000 7f1996d9d640 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 0x5569bc447780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.858+0000 7f1996d9d640 1 -- 172.21.15.45:0/732524200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569bc4626d0 con 0x5569bc44fe80 2024-09-10T10:07:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.858+0000 7f1996d9d640 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d630 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1988005380 tx=0x7f1988005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d798edc593ed32f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.859+0000 7f197ffff640 1 -- 172.21.15.45:0/732524200 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1988017050 con 0x5569bc44fe80 2024-09-10T10:07:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.859+0000 7f197ffff640 1 -- 172.21.15.45:0/732524200 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1988033070 con 0x5569bc44fe80 2024-09-10T10:07:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.859+0000 7f197ffff640 1 -- 172.21.15.45:0/732524200 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19880047c0 con 0x5569bc44fe80 2024-09-10T10:07:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.860+0000 7f199fcd6740 1 -- 172.21.15.45:0/732524200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 msgr2=0x5569bc44d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.860+0000 7f199fcd6740 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d630 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1988005380 tx=0x7f1988005f70 comp rx=0 tx=0).stop 2024-09-10T10:07:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.860+0000 7f199fcd6740 1 -- 172.21.15.45:0/732524200 shutdown_connections 2024-09-10T10:07:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.860+0000 7f199fcd6740 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc44fe80 0x5569bc44d630 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.860+0000 7f199fcd6740 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc4455e0 0x5569bc440170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.860+0000 7f199fcd6740 1 --2- 172.21.15.45:0/732524200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 0x5569bc447780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.861+0000 7f199fcd6740 1 -- 172.21.15.45:0/732524200 >> 172.21.15.45:0/732524200 conn(0x5569bc441230 msgr2=0x5569bc43fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.861+0000 7f199fcd6740 1 -- 172.21.15.45:0/732524200 shutdown_connections 2024-09-10T10:07:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.861+0000 7f199fcd6740 1 -- 172.21.15.45:0/732524200 wait complete. 2024-09-10T10:07:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.862+0000 7f199fcd6740 1 -- start start 2024-09-10T10:07:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.862+0000 7f199fcd6740 1 Processor -- start 2024-09-10T10:07:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.862+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 0x5569bc458190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.862+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc4586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.862+0000 7f199fcd6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc44fe80 0x5569bc453510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f199fcd6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19981bb7f0 con 0x5569bc4455e0 2024-09-10T10:07:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f199fcd6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f19981bcc60 con 0x5569bc444cc0 2024-09-10T10:07:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f199fcd6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f19981bbec0 con 0x5569bc44fe80 2024-09-10T10:07:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f199fcd6740 10 client.0 _ll_register_callbacks cb 0x7f199000eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc4586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc4586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50450/0 (socket says 172.21.15.45:50450) 2024-09-10T10:07:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 -- 172.21.15.45:0/3389125297 learned_addr learned my addr 172.21.15.45:0/3389125297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1996d9d640 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc44fe80 0x5569bc453510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 -- 172.21.15.45:0/3389125297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc44fe80 msgr2=0x5569bc453510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc44fe80 0x5569bc453510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 -- 172.21.15.45:0/3389125297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 msgr2=0x5569bc458190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:07:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 0x5569bc458190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.863+0000 7f1995d9b640 1 -- 172.21.15.45:0/3389125297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569bc453d40 con 0x5569bc4455e0 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.864+0000 7f1995d9b640 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc4586d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f198c009900 tx=0x7f198c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b51adb1d17eb98a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.864+0000 7f197dffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.864+0000 7f197dffb640 1 -- 172.21.15.45:0/3389125297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c035040 con 0x5569bc4455e0 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.864+0000 7f197dffb640 1 -- 172.21.15.45:0/3389125297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f198c0035f0 con 0x5569bc4455e0 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.864+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1988033070 con 0x5569bc4455e0 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.865+0000 7f197dffb640 1 -- 172.21.15.45:0/3389125297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f198c0037f0 con 0x5569bc4455e0 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.865+0000 7f199fcd6740 10 client.15405 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.865+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5569bc4ca4d0 con 0x5569bc4455e0 2024-09-10T10:07:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f197dffb640 1 -- 172.21.15.45:0/3389125297 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198c0315b0 con 0x5569bc4455e0 2024-09-10T10:07:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f197dffb640 1 -- 172.21.15.45:0/3389125297 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f198c03c440 con 0x5569bc4455e0 2024-09-10T10:07:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f199fcd6740 10 client.15405 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f199fcd6740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f199fcd6740 20 client.15405 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f199fcd6740 4 client.15405 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:07:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.866+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f1988033070 con 0x5569bc45fd00 2024-09-10T10:07:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.868+0000 7f199659c640 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.872+0000 7f199659c640 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4ccc90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf163da6fe36fca server_cookie=978522dded561523 in_seq=0 out_seq=0 2024-09-10T10:07:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:37.873+0000 7f197dffb640 10 client.15405 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:37 smithi162 ceph-mon[34963]: pgmap v135: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 763 MiB/s wr, 1.80k op/s 2024-09-10T10:07:37.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:37 smithi045 ceph-mon[31114]: pgmap v135: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 763 MiB/s wr, 1.80k op/s 2024-09-10T10:07:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:37 smithi047 ceph-mon[35244]: pgmap v135: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 763 MiB/s wr, 1.80k op/s 2024-09-10T10:07:39.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:39 smithi162 ceph-mon[34963]: pgmap v136: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 830 MiB/s wr, 1.87k op/s 2024-09-10T10:07:39.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:39 smithi045 ceph-mon[31114]: pgmap v136: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 830 MiB/s wr, 1.87k op/s 2024-09-10T10:07:39.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:39 smithi047 ceph-mon[35244]: pgmap v136: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 830 MiB/s wr, 1.87k op/s 2024-09-10T10:07:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.212+0000 7f197dffb640 1 -- 172.21.15.45:0/3389125297 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f1988033070 con 0x5569bc45fd00 2024-09-10T10:07:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.212+0000 7f197dffb640 10 client.15405 handle_command_reply: tid=0 2024-09-10T10:07:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.212+0000 7f199fcd6740 1 client.15405 shutdown 2024-09-10T10:07:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 msgr2=0x5569bc4586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc4586d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f198c009900 tx=0x7f198c004040 comp rx=0 tx=0).stop 2024-09-10T10:07:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 shutdown_connections 2024-09-10T10:07:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5569bc45fd00 0x5569bc4ccc90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5569bc44fe80 0x5569bc453510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5569bc4455e0 0x5569bc4586d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 --2- 172.21.15.45:0/3389125297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569bc444cc0 0x5569bc458190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.213+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 >> 172.21.15.45:0/3389125297 conn(0x5569bc441230 msgr2=0x5569bc440280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.214+0000 7f197dffb640 0 client.15405 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.214+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 shutdown_connections 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.214+0000 7f199fcd6740 1 -- 172.21.15.45:0/3389125297 wait complete. 2024-09-10T10:07:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.215+0000 7f199fcd6740 20 client.15405 trim_cache size 0 max 16384 2024-09-10T10:07:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:07:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:07:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:07:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:07:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.218+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f19800a2da0 msgr2=0x7f19800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.218+0000 7f199cf3b640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f19800a2da0 0x7f19800a5260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f198801c000 tx=0x7f198801e000 comp rx=0 tx=0).stop 2024-09-10T10:07:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.219+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 msgr2=0x7f1998112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.219+0000 7f199cf3b640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 0x7f1998112ab0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f199000ebc0 tx=0x7f199000c6a0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.220+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 shutdown_connections 2024-09-10T10:07:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.221+0000 7f199cf3b640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f19981135d0 0x7f19981b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.221+0000 7f199cf3b640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f19800a2da0 0x7f19800a5260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.221+0000 7f199cf3b640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1998114ab0 0x7f1998112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.221+0000 7f199cf3b640 1 --2- 172.21.15.45:0/2882805387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19981091f0 0x7f1998112ab0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.221+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 >> 172.21.15.45:0/2882805387 conn(0x7f199806cc60 msgr2=0x7f1998070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.222+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 shutdown_connections 2024-09-10T10:07:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:40.222+0000 7f199cf3b640 1 -- 172.21.15.45:0/2882805387 wait complete. 2024-09-10T10:07:41.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:41 smithi045 ceph-mon[31114]: pgmap v137: 97 pgs: 97 active+clean; 40 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 929 MiB/s wr, 1.93k op/s 2024-09-10T10:07:41.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:41 smithi047 ceph-mon[35244]: pgmap v137: 97 pgs: 97 active+clean; 40 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 929 MiB/s wr, 1.93k op/s 2024-09-10T10:07:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:41 smithi162 ceph-mon[34963]: pgmap v137: 97 pgs: 97 active+clean; 40 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 929 MiB/s wr, 1.93k op/s 2024-09-10T10:07:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:43 smithi162 ceph-mon[34963]: pgmap v138: 97 pgs: 97 active+clean; 40 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 765 MiB/s wr, 1.55k op/s 2024-09-10T10:07:44.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:43 smithi045 ceph-mon[31114]: pgmap v138: 97 pgs: 97 active+clean; 40 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 765 MiB/s wr, 1.55k op/s 2024-09-10T10:07:44.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:43 smithi047 ceph-mon[35244]: pgmap v138: 97 pgs: 97 active+clean; 40 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 765 MiB/s wr, 1.55k op/s 2024-09-10T10:07:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:45 smithi162 ceph-mon[34963]: pgmap v139: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 981 MiB/s wr, 1.81k op/s 2024-09-10T10:07:46.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:45 smithi045 ceph-mon[31114]: pgmap v139: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 981 MiB/s wr, 1.81k op/s 2024-09-10T10:07:46.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:45 smithi047 ceph-mon[35244]: pgmap v139: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 981 MiB/s wr, 1.81k op/s 2024-09-10T10:07:47.053 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:46 smithi045 ceph-mon[31114]: pgmap v140: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 790 MiB/s wr, 1.09k op/s 2024-09-10T10:07:47.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:46 smithi162 ceph-mon[34963]: pgmap v140: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 790 MiB/s wr, 1.09k op/s 2024-09-10T10:07:47.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:46 smithi047 ceph-mon[35244]: pgmap v140: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 790 MiB/s wr, 1.09k op/s 2024-09-10T10:07:49.402 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:48 smithi162 ceph-mon[34963]: pgmap v141: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 816 MiB/s wr, 1.19k op/s 2024-09-10T10:07:49.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:48 smithi045 ceph-mon[31114]: pgmap v141: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 816 MiB/s wr, 1.19k op/s 2024-09-10T10:07:49.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:48 smithi047 ceph-mon[35244]: pgmap v141: 97 pgs: 97 active+clean; 40 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 816 MiB/s wr, 1.19k op/s 2024-09-10T10:07:51.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:51 smithi162 ceph-mon[34963]: pgmap v142: 97 pgs: 97 active+clean; 40 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 914 MiB/s wr, 1.61k op/s 2024-09-10T10:07:51.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:51 smithi045 ceph-mon[31114]: pgmap v142: 97 pgs: 97 active+clean; 40 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 914 MiB/s wr, 1.61k op/s 2024-09-10T10:07:51.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:51 smithi047 ceph-mon[35244]: pgmap v142: 97 pgs: 97 active+clean; 40 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 914 MiB/s wr, 1.61k op/s 2024-09-10T10:07:53.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:53 smithi162 ceph-mon[34963]: pgmap v143: 97 pgs: 97 active+clean; 40 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 747 MiB/s wr, 1.48k op/s 2024-09-10T10:07:53.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:53 smithi045 ceph-mon[31114]: pgmap v143: 97 pgs: 97 active+clean; 40 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 747 MiB/s wr, 1.48k op/s 2024-09-10T10:07:53.411 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:53 smithi047 ceph-mon[35244]: pgmap v143: 97 pgs: 97 active+clean; 40 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 747 MiB/s wr, 1.48k op/s 2024-09-10T10:07:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.578+0000 7f2e545b7640 1 -- 172.21.15.45:0/2016553544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1095d0 msgr2=0x7f2e4c109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.579+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2016553544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1095d0 0x7f2e4c109a10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009be0 tx=0x7f2e4402a710 comp rx=0 tx=0).stop 2024-09-10T10:07:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.580+0000 7f2e545b7640 1 -- 172.21.15.45:0/2016553544 shutdown_connections 2024-09-10T10:07:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.580+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2016553544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e4c11b140 0x7f2e4c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.580+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2016553544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1095d0 0x7f2e4c109a10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.580+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2016553544 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c1091f0 0x7f2e4c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.580+0000 7f2e545b7640 1 -- 172.21.15.45:0/2016553544 >> 172.21.15.45:0/2016553544 conn(0x7f2e4c06cc60 msgr2=0x7f2e4c06d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.580+0000 7f2e545b7640 1 -- 172.21.15.45:0/2016553544 shutdown_connections 2024-09-10T10:07:53.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.580+0000 7f2e545b7640 1 -- 172.21.15.45:0/2016553544 wait complete. 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.581+0000 7f2e545b7640 1 Processor -- start 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.581+0000 7f2e545b7640 1 -- start start 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.582+0000 7f2e545b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1091f0 0x7f2e4c11a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.582+0000 7f2e545b7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e4c1095d0 0x7f2e4c1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.582+0000 7f2e545b7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c11b140 0x7f2e4c113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.582+0000 7f2e545b7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c120bc0 con 0x7f2e4c1091f0 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.582+0000 7f2e545b7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c120a40 con 0x7f2e4c1095d0 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.582+0000 7f2e545b7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c120d40 con 0x7f2e4c11b140 2024-09-10T10:07:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c11b140 0x7f2e4c113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c11b140 0x7f2e4c113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:34326/0 (socket says 172.21.15.45:34326) 2024-09-10T10:07:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 -- 172.21.15.45:0/2638341 learned_addr learned my addr 172.21.15.45:0/2638341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e51b52640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e4c1095d0 0x7f2e4c1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52353640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1091f0 0x7f2e4c11a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 -- 172.21.15.45:0/2638341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e4c1095d0 msgr2=0x7f2e4c1138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e4c1095d0 0x7f2e4c1138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 -- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1091f0 msgr2=0x7f2e4c11a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1091f0 0x7f2e4c11a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.583+0000 7f2e52b54640 1 -- 172.21.15.45:0/2638341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c114520 con 0x7f2e4c11b140 2024-09-10T10:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.584+0000 7f2e52b54640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c11b140 0x7f2e4c113df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800e940 tx=0x7f2e4800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.584+0000 7f2e437fe640 1 -- 172.21.15.45:0/2638341 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4800ccc0 con 0x7f2e4c11b140 2024-09-10T10:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.585+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c118350 con 0x7f2e4c11b140 2024-09-10T10:07:53.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.585+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c118870 con 0x7f2e4c11b140 2024-09-10T10:07:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.594+0000 7f2e5742c740 1 Processor -- start 2024-09-10T10:07:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.594+0000 7f2e5742c740 1 -- start start 2024-09-10T10:07:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.596+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.596+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eb5e0 0x55c3d01f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.596+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.596+0000 7f2e5742c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c3d014f2f0 con 0x55c3d01f5e80 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e5742c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c3d00f7270 con 0x55c3d01eb5e0 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e5742c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c3d014b2b0 con 0x55c3d01eacc0 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e51b52640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e51b52640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01eb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:34328/0 (socket says 172.21.15.45:34328) 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e51b52640 1 -- 172.21.15.45:0/2903352480 learned_addr learned my addr 172.21.15.45:0/2903352480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e437fe640 1 -- 172.21.15.45:0/2638341 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e48005500 con 0x7f2e4c11b140 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e437fe640 1 -- 172.21.15.45:0/2638341 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48010690 con 0x7f2e4c11b140 2024-09-10T10:07:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.597+0000 7f2e437fe640 1 -- 172.21.15.45:0/2638341 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2e480108f0 con 0x7f2e4c11b140 2024-09-10T10:07:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.598+0000 7f2e51b52640 1 -- 172.21.15.45:0/2903352480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eb5e0 msgr2=0x55c3d01f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.598+0000 7f2e51b52640 1 --2- 172.21.15.45:0/2903352480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eb5e0 0x55c3d01f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.598+0000 7f2e51b52640 1 -- 172.21.15.45:0/2903352480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 msgr2=0x55c3d01f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.598+0000 7f2e51b52640 1 --2- 172.21.15.45:0/2903352480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.598+0000 7f2e51b52640 1 -- 172.21.15.45:0/2903352480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d00f6340 con 0x55c3d01eacc0 2024-09-10T10:07:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.604+0000 7f2e51b52640 1 --2- 172.21.15.45:0/2903352480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01eb0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009be0 tx=0x7f2e44030640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b81af9e705b8f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.605+0000 7f2e437fe640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f2e380a2e60 0x7f2e380a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.606+0000 7f2e437fe640 1 -- 172.21.15.45:0/2638341 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f2e480d0780 con 0x7f2e4c11b140 2024-09-10T10:07:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.606+0000 7f2e40bfe640 1 -- 172.21.15.45:0/2903352480 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44043070 con 0x55c3d01eacc0 2024-09-10T10:07:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.606+0000 7f2e40bfe640 1 -- 172.21.15.45:0/2903352480 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e44030e60 con 0x55c3d01eacc0 2024-09-10T10:07:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.608+0000 7f2e5742c740 1 -- 172.21.15.45:0/2903352480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 msgr2=0x55c3d01eb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.608+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2903352480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01eb0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009be0 tx=0x7f2e44030640 comp rx=0 tx=0).stop 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.610+0000 7f2e52353640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f2e380a2e60 0x7f2e380a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.611+0000 7f2e5742c740 1 -- 172.21.15.45:0/2903352480 shutdown_connections 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.611+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2903352480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.611+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2903352480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eb5e0 0x55c3d01f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.611+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2903352480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01eb0a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.611+0000 7f2e5742c740 1 -- 172.21.15.45:0/2903352480 >> 172.21.15.45:0/2903352480 conn(0x55c3d01e7230 msgr2=0x55c3d01e7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.611+0000 7f2e5742c740 1 -- 172.21.15.45:0/2903352480 shutdown_connections 2024-09-10T10:07:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.611+0000 7f2e5742c740 1 -- 172.21.15.45:0/2903352480 wait complete. 2024-09-10T10:07:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.612+0000 7f2e5742c740 1 -- start start 2024-09-10T10:07:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.616+0000 7f2e5742c740 1 Processor -- start 2024-09-10T10:07:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.617+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 0x55c3d01f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.617+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 0x55c3d01f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.617+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.618+0000 7f2e5742c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c3d01f8aa0 con 0x55c3d01f5e80 2024-09-10T10:07:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.618+0000 7f2e5742c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c3d00d8290 con 0x55c3d01eacc0 2024-09-10T10:07:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.618+0000 7f2e5742c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c3d01f8c20 con 0x55c3d01eb5e0 2024-09-10T10:07:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.618+0000 7f2e5742c740 10 client.0 _ll_register_callbacks cb 0x55c3d0109140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.620+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.620+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41434/0 (socket says 172.21.15.45:41434) 2024-09-10T10:07:53.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.620+0000 7f2e52b54640 1 -- 172.21.15.45:0/3403628703 learned_addr learned my addr 172.21.15.45:0/3403628703 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.620+0000 7f2e51b52640 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 0x55c3d01f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.621+0000 7f2e52353640 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 0x55c3d01f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.621+0000 7f2e52b54640 1 -- 172.21.15.45:0/3403628703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 msgr2=0x55c3d01f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.621+0000 7f2e52b54640 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 0x55c3d01f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.621+0000 7f2e52b54640 1 -- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 msgr2=0x55c3d01f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.621+0000 7f2e52b54640 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 0x55c3d01f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.621+0000 7f2e52b54640 1 -- 172.21.15.45:0/3403628703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3cfe62210 con 0x55c3d01f5e80 2024-09-10T10:07:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.622+0000 7f2e52353640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f2e380a2e60 0x7f2e380a5320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c3d01e8090 tx=0x7f2e3c008040 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.623+0000 7f2e52b54640 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f37a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e48004ba0 tx=0x7f2e4809d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c28ca01a329d987 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.623+0000 7f2e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:07:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.624+0000 7f2e35ffb640 1 -- 172.21.15.45:0/3403628703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480a5960 con 0x55c3d01f5e80 2024-09-10T10:07:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.624+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e44030e60 con 0x55c3d01f5e80 2024-09-10T10:07:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.624+0000 7f2e35ffb640 1 -- 172.21.15.45:0/3403628703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f2e4809cf50 con 0x55c3d01f5e80 2024-09-10T10:07:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.625+0000 7f2e35ffb640 1 -- 172.21.15.45:0/3403628703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e4809d150 con 0x55c3d01f5e80 2024-09-10T10:07:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.625+0000 7f2e5742c740 10 client.15411 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.625+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c3cff2e620 con 0x55c3d01f5e80 2024-09-10T10:07:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.627+0000 7f2e35ffb640 1 -- 172.21.15.45:0/3403628703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e44030e60 con 0x55c3d01f5e80 2024-09-10T10:07:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.627+0000 7f2e35ffb640 1 -- 172.21.15.45:0/3403628703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2e480046c0 con 0x55c3d01f5e80 2024-09-10T10:07:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.628+0000 7f2e5742c740 10 client.15411 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.628+0000 7f2e5742c740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.628+0000 7f2e5742c740 20 client.15411 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.628+0000 7f2e5742c740 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02062c0 0x55c3d0208720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.628+0000 7f2e5742c740 4 client.15411 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:07:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.628+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3d015ca20 con 0x55c3d02062c0 2024-09-10T10:07:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.631+0000 7f2e52353640 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02062c0 0x55c3d0208720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.646+0000 7f2e35ffb640 10 client.15411 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.646+0000 7f2e52353640 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02062c0 0x55c3d0208720 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b475310021fe251 server_cookie=94f9da3e6ae975ec in_seq=0 out_seq=0 2024-09-10T10:07:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.648+0000 7f2e35ffb640 1 -- 172.21.15.45:0/3403628703 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3d015ca20 con 0x55c3d02062c0 2024-09-10T10:07:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.648+0000 7f2e35ffb640 10 client.15411 handle_command_reply: tid=0 2024-09-10T10:07:53.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.648+0000 7f2e5742c740 1 client.15411 shutdown 2024-09-10T10:07:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 msgr2=0x55c3d01f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f37a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e48004ba0 tx=0x7f2e4809d3f0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 shutdown_connections 2024-09-10T10:07:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02062c0 0x55c3d0208720 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01f37a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 0x55c3d01f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 --2- 172.21.15.45:0/3403628703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 0x55c3d01f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 >> 172.21.15.45:0/3403628703 conn(0x55c3d01e7230 msgr2=0x55c3d01ecf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:53.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.649+0000 7f2e35ffb640 0 client.15411 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.652+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 shutdown_connections 2024-09-10T10:07:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.652+0000 7f2e5742c740 1 -- 172.21.15.45:0/3403628703 wait complete. 2024-09-10T10:07:53.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.652+0000 7f2e5742c740 20 client.15411 trim_cache size 0 max 16384 2024-09-10T10:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.687+0000 7f2e5742c740 1 Processor -- start 2024-09-10T10:07:53.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.687+0000 7f2e5742c740 1 -- start start 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01eb5e0 0x55c3d01e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01f5e80 0x55c3d020e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e5742c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3000c810 con 0x55c3d01eb5e0 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e5742c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3000f0b0 con 0x55c3d01f5e80 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e5742c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3000cea0 con 0x55c3d01eacc0 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01f5e80 0x55c3d020e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e51b52640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01eb5e0 0x55c3d01e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01f5e80 0x55c3d020e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:53666/0 (socket says 172.21.15.45:53666) 2024-09-10T10:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.688+0000 7f2e52b54640 1 -- 172.21.15.45:0/2590992361 learned_addr learned my addr 172.21.15.45:0/2590992361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.689+0000 7f2e51b52640 1 -- 172.21.15.45:0/2590992361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 msgr2=0x55c3d01f35d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:07:53.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.689+0000 7f2e51b52640 1 --2- 172.21.15.45:0/2590992361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01f35d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.689+0000 7f2e51b52640 1 -- 172.21.15.45:0/2590992361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01f5e80 msgr2=0x55c3d020e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.689+0000 7f2e51b52640 1 --2- 172.21.15.45:0/2590992361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01f5e80 0x55c3d020e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.689+0000 7f2e51b52640 1 -- 172.21.15.45:0/2590992361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d01f40a0 con 0x55c3d01eb5e0 2024-09-10T10:07:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.689+0000 7f2e51b52640 1 --2- 172.21.15.45:0/2590992361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01eb5e0 0x55c3d01e6170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2e44004af0 tx=0x7f2e44005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8622ba738b755505 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.690+0000 7f2e37fff640 1 -- 172.21.15.45:0/2590992361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e44043040 con 0x55c3d01eb5e0 2024-09-10T10:07:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.690+0000 7f2e37fff640 1 -- 172.21.15.45:0/2590992361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e44030e60 con 0x55c3d01eb5e0 2024-09-10T10:07:53.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.690+0000 7f2e37fff640 1 -- 172.21.15.45:0/2590992361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4403f970 con 0x55c3d01eb5e0 2024-09-10T10:07:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 -- 172.21.15.45:0/2590992361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01eb5e0 msgr2=0x55c3d01e6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2590992361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01eb5e0 0x55c3d01e6170 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2e44004af0 tx=0x7f2e44005bd0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 -- 172.21.15.45:0/2590992361 shutdown_connections 2024-09-10T10:07:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2590992361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01f5e80 0x55c3d020e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2590992361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01eb5e0 0x55c3d01e6170 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 --2- 172.21.15.45:0/2590992361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eacc0 0x55c3d01f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 -- 172.21.15.45:0/2590992361 >> 172.21.15.45:0/2590992361 conn(0x55c3d01e7230 msgr2=0x55c3d01e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 -- 172.21.15.45:0/2590992361 shutdown_connections 2024-09-10T10:07:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.691+0000 7f2e5742c740 1 -- 172.21.15.45:0/2590992361 wait complete. 2024-09-10T10:07:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.692+0000 7f2e5742c740 1 -- start start 2024-09-10T10:07:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.692+0000 7f2e5742c740 1 Processor -- start 2024-09-10T10:07:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.693+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 0x55c3d01fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.693+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 0x55c3d01fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.693+0000 7f2e5742c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.693+0000 7f2e5742c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3000d570 con 0x55c3d01f5e80 2024-09-10T10:07:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.693+0000 7f2e5742c740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3000e9e0 con 0x55c3d01eacc0 2024-09-10T10:07:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.693+0000 7f2e5742c740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3000dc40 con 0x55c3d01eb5e0 2024-09-10T10:07:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.693+0000 7f2e5742c740 10 client.0 _ll_register_callbacks cb 0x7f2e480056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41462/0 (socket says 172.21.15.45:41462) 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 -- 172.21.15.45:0/4291384201 learned_addr learned my addr 172.21.15.45:0/4291384201 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 -- 172.21.15.45:0/4291384201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 msgr2=0x55c3d01fe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 0x55c3d01fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 -- 172.21.15.45:0/4291384201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 msgr2=0x55c3d01fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 0x55c3d01fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.694+0000 7f2e52b54640 1 -- 172.21.15.45:0/4291384201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d01efeb0 con 0x55c3d01f5e80 2024-09-10T10:07:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.695+0000 7f2e52b54640 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01ef710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2e4809d5a0 tx=0x7f2e4800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1222198cb12162d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:07:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.695+0000 7f2e35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:07:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.695+0000 7f2e35ffb640 1 -- 172.21.15.45:0/4291384201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480a5960 con 0x55c3d01f5e80 2024-09-10T10:07:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.695+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e44030e60 con 0x55c3d01f5e80 2024-09-10T10:07:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.695+0000 7f2e35ffb640 1 -- 172.21.15.45:0/4291384201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f2e480a54d0 con 0x55c3d01f5e80 2024-09-10T10:07:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.696+0000 7f2e35ffb640 1 -- 172.21.15.45:0/4291384201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2e480a56d0 con 0x55c3d01f5e80 2024-09-10T10:07:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.696+0000 7f2e5742c740 10 client.15417 fetch_fsmap learned FSMap version 19 2024-09-10T10:07:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.696+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c3d01fdc60 con 0x55c3d01f5e80 2024-09-10T10:07:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.696+0000 7f2e35ffb640 1 -- 172.21.15.45:0/4291384201 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e44030e60 con 0x55c3d01f5e80 2024-09-10T10:07:53.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.696+0000 7f2e35ffb640 1 -- 172.21.15.45:0/4291384201 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f2e4809bbb0 con 0x55c3d01f5e80 2024-09-10T10:07:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.697+0000 7f2e5742c740 10 client.15417 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:07:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.697+0000 7f2e5742c740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:07:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.697+0000 7f2e5742c740 20 client.15417 populate_metadata read hostname 'smithi045' 2024-09-10T10:07:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.697+0000 7f2e5742c740 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02713a0 0x55c3d0273820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:07:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.697+0000 7f2e5742c740 4 client.15417 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:07:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.697+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3d0156d70 con 0x55c3d02713a0 2024-09-10T10:07:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.698+0000 7f2e52353640 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02713a0 0x55c3d0273820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:07:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.699+0000 7f2e52353640 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02713a0 0x55c3d0273820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd1f6e184070325c server_cookie=1c2d8405efb79a4a in_seq=0 out_seq=0 2024-09-10T10:07:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:53.699+0000 7f2e35ffb640 10 client.15417 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.552+0000 7f2e35ffb640 1 -- 172.21.15.45:0/4291384201 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3d0156d70 con 0x55c3d02713a0 2024-09-10T10:07:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.552+0000 7f2e35ffb640 10 client.15417 handle_command_reply: tid=0 2024-09-10T10:07:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.552+0000 7f2e5742c740 1 client.15417 shutdown 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 msgr2=0x55c3d01ef710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01ef710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2e4809d5a0 tx=0x7f2e4800ef30 comp rx=0 tx=0).stop 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 shutdown_connections 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c3d02713a0 0x55c3d0273820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3d01f5e80 0x55c3d01ef710 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c3d01eb5e0 0x55c3d01fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 --2- 172.21.15.45:0/4291384201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c3d01eacc0 0x55c3d01fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 >> 172.21.15.45:0/4291384201 conn(0x55c3d01e7230 msgr2=0x55c3d01e6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.558+0000 7f2e35ffb640 0 client.15417 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:07:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.560+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 shutdown_connections 2024-09-10T10:07:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.560+0000 7f2e5742c740 1 -- 172.21.15.45:0/4291384201 wait complete. 2024-09-10T10:07:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.560+0000 7f2e5742c740 20 client.15417 trim_cache size 0 max 16384 2024-09-10T10:07:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:07:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:07:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:07:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:07:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.571+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f2e380a2e60 msgr2=0x7f2e380a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.571+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f2e380a2e60 0x7f2e380a5320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55c3d01e8090 tx=0x7f2e3c008040 comp rx=0 tx=0).stop 2024-09-10T10:07:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.571+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c11b140 msgr2=0x7f2e4c113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:07:55.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.571+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c11b140 0x7f2e4c113df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800e940 tx=0x7f2e4800ee00 comp rx=0 tx=0).stop 2024-09-10T10:07:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.576+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 shutdown_connections 2024-09-10T10:07:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.576+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2e4c11b140 0x7f2e4c113df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.576+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f2e380a2e60 0x7f2e380a5320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.576+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e4c1095d0 0x7f2e4c1138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.576+0000 7f2e545b7640 1 --2- 172.21.15.45:0/2638341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e4c1091f0 0x7f2e4c11a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:07:55.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.576+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 >> 172.21.15.45:0/2638341 conn(0x7f2e4c06cc60 msgr2=0x7f2e4c070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:07:55.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.578+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 shutdown_connections 2024-09-10T10:07:55.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:07:55.581+0000 7f2e545b7640 1 -- 172.21.15.45:0/2638341 wait complete. 2024-09-10T10:07:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:55 smithi162 ceph-mon[34963]: pgmap v144: 97 pgs: 97 active+clean; 40 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 988 MiB/s wr, 1.74k op/s 2024-09-10T10:07:55.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:55 smithi045 ceph-mon[31114]: pgmap v144: 97 pgs: 97 active+clean; 40 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 988 MiB/s wr, 1.74k op/s 2024-09-10T10:07:55.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:55 smithi047 ceph-mon[35244]: pgmap v144: 97 pgs: 97 active+clean; 40 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 988 MiB/s wr, 1.74k op/s 2024-09-10T10:07:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:57 smithi162 ceph-mon[34963]: pgmap v145: 97 pgs: 97 active+clean; 40 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 816 MiB/s wr, 1.51k op/s 2024-09-10T10:07:57.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:57 smithi045 ceph-mon[31114]: pgmap v145: 97 pgs: 97 active+clean; 40 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 816 MiB/s wr, 1.51k op/s 2024-09-10T10:07:57.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:57 smithi047 ceph-mon[35244]: pgmap v145: 97 pgs: 97 active+clean; 40 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 816 MiB/s wr, 1.51k op/s 2024-09-10T10:07:59.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:07:59 smithi162 ceph-mon[34963]: pgmap v146: 97 pgs: 97 active+clean; 40 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 857 MiB/s wr, 1.80k op/s 2024-09-10T10:07:59.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:07:59 smithi045 ceph-mon[31114]: pgmap v146: 97 pgs: 97 active+clean; 40 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 857 MiB/s wr, 1.80k op/s 2024-09-10T10:07:59.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:07:59 smithi047 ceph-mon[35244]: pgmap v146: 97 pgs: 97 active+clean; 40 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 857 MiB/s wr, 1.80k op/s 2024-09-10T10:08:02.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:01 smithi162 ceph-mon[34963]: pgmap v147: 97 pgs: 97 active+clean; 40 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 918 MiB/s wr, 2.06k op/s 2024-09-10T10:08:02.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:01 smithi045 ceph-mon[31114]: pgmap v147: 97 pgs: 97 active+clean; 40 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 918 MiB/s wr, 2.06k op/s 2024-09-10T10:08:02.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:01 smithi047 ceph-mon[35244]: pgmap v147: 97 pgs: 97 active+clean; 40 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 918 MiB/s wr, 2.06k op/s 2024-09-10T10:08:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:03 smithi162 ceph-mon[34963]: pgmap v148: 97 pgs: 97 active+clean; 40 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 758 MiB/s wr, 1.66k op/s 2024-09-10T10:08:04.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:03 smithi045 ceph-mon[31114]: pgmap v148: 97 pgs: 97 active+clean; 40 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 758 MiB/s wr, 1.66k op/s 2024-09-10T10:08:04.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:03 smithi047 ceph-mon[35244]: pgmap v148: 97 pgs: 97 active+clean; 40 GiB data, 216 GiB used, 857 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 758 MiB/s wr, 1.66k op/s 2024-09-10T10:08:06.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:05 smithi162 ceph-mon[34963]: pgmap v149: 97 pgs: 97 active+clean; 40 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 1001 MiB/s wr, 2.24k op/s 2024-09-10T10:08:06.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:05 smithi045 ceph-mon[31114]: pgmap v149: 97 pgs: 97 active+clean; 40 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 1001 MiB/s wr, 2.24k op/s 2024-09-10T10:08:06.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:05 smithi047 ceph-mon[35244]: pgmap v149: 97 pgs: 97 active+clean; 40 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 1001 MiB/s wr, 2.24k op/s 2024-09-10T10:08:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:07 smithi162 ceph-mon[34963]: pgmap v150: 97 pgs: 97 active+clean; 40 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 807 MiB/s wr, 2.08k op/s 2024-09-10T10:08:08.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:07 smithi045 ceph-mon[31114]: pgmap v150: 97 pgs: 97 active+clean; 40 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 807 MiB/s wr, 2.08k op/s 2024-09-10T10:08:08.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:07 smithi047 ceph-mon[35244]: pgmap v150: 97 pgs: 97 active+clean; 40 GiB data, 220 GiB used, 853 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 807 MiB/s wr, 2.08k op/s 2024-09-10T10:08:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 -- 172.21.15.45:0/3049922015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a81009d0 msgr2=0x7f17a8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 --2- 172.21.15.45:0/3049922015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a81009d0 0x7f17a8100e30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1794009930 tx=0x7f179402a420 comp rx=0 tx=0).stop 2024-09-10T10:08:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 -- 172.21.15.45:0/3049922015 shutdown_connections 2024-09-10T10:08:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 --2- 172.21.15.45:0/3049922015 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a8101370 0x7f17a810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 --2- 172.21.15.45:0/3049922015 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a81009d0 0x7f17a8100e30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 --2- 172.21.15.45:0/3049922015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a81069e0 0x7f17a8106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 -- 172.21.15.45:0/3049922015 >> 172.21.15.45:0/3049922015 conn(0x7f17a80fc6b0 msgr2=0x7f17a80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 -- 172.21.15.45:0/3049922015 shutdown_connections 2024-09-10T10:08:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.797+0000 7f17af754640 1 -- 172.21.15.45:0/3049922015 wait complete. 2024-09-10T10:08:08.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.798+0000 7f17af754640 1 Processor -- start 2024-09-10T10:08:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.799+0000 7f17af754640 1 -- start start 2024-09-10T10:08:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.799+0000 7f17af754640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a81009d0 0x7f17a810e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.799+0000 7f17af754640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a8101370 0x7f17a810e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.799+0000 7f17af754640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a81069e0 0x7f17a810edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.799+0000 7f17af754640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8111f60 con 0x7f17a8101370 2024-09-10T10:08:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.799+0000 7f17af754640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8111de0 con 0x7f17a81009d0 2024-09-10T10:08:08.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.799+0000 7f17af754640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81120e0 con 0x7f17a81069e0 2024-09-10T10:08:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a8101370 0x7f17a810e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a8101370 0x7f17a810e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35936/0 (socket says 172.21.15.45:35936) 2024-09-10T10:08:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 -- 172.21.15.45:0/601656272 learned_addr learned my addr 172.21.15.45:0/601656272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a81009d0 0x7f17a810e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 -- 172.21.15.45:0/601656272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a81069e0 msgr2=0x7f17a810edb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a81069e0 0x7f17a810edb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 -- 172.21.15.45:0/601656272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a81009d0 msgr2=0x7f17a810e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a81009d0 0x7f17a810e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.800+0000 7f17accef640 1 -- 172.21.15.45:0/601656272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a81a6900 con 0x7f17a8101370 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.801+0000 7f17accef640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a8101370 0x7f17a810e870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f179402a940 tx=0x7f1794003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:08.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.801+0000 7f17a67fc640 1 -- 172.21.15.45:0/601656272 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179403f070 con 0x7f17a8101370 2024-09-10T10:08:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.801+0000 7f17a67fc640 1 -- 172.21.15.45:0/601656272 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1794031c90 con 0x7f17a8101370 2024-09-10T10:08:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.801+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a81a6b90 con 0x7f17a8101370 2024-09-10T10:08:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.801+0000 7f17a67fc640 1 -- 172.21.15.45:0/601656272 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794035030 con 0x7f17a8101370 2024-09-10T10:08:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.801+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a81a70d0 con 0x7f17a8101370 2024-09-10T10:08:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.805+0000 7f17b25c8740 1 Processor -- start 2024-09-10T10:08:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.806+0000 7f17b25c8740 1 -- start start 2024-09-10T10:08:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.807+0000 7f17a67fc640 1 -- 172.21.15.45:0/601656272 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1794042070 con 0x7f17a8101370 2024-09-10T10:08:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17b25c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56440e93e2f0 con 0x56440e9e4cb0 2024-09-10T10:08:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17b25c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56440e8e6270 con 0x56440e9da410 2024-09-10T10:08:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17b25c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56440e93a2b0 con 0x56440e9d9af0 2024-09-10T10:08:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17accef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17accef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35938/0 (socket says 172.21.15.45:35938) 2024-09-10T10:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.808+0000 7f17accef640 1 -- 172.21.15.45:0/523013849 learned_addr learned my addr 172.21.15.45:0/523013849 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.809+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.809+0000 7f17adcf1640 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.810+0000 7f17a67fc640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2e20 0x7f178c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.810+0000 7f17a67fc640 1 -- 172.21.15.45:0/601656272 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f17940efc00 con 0x7f17a8101370 2024-09-10T10:08:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.810+0000 7f17adcf1640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2e20 0x7f178c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.811+0000 7f17accef640 1 -- 172.21.15.45:0/523013849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 msgr2=0x56440e9d9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.811+0000 7f17accef640 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9d9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.811+0000 7f17accef640 1 -- 172.21.15.45:0/523013849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 msgr2=0x56440e9e4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.811+0000 7f17accef640 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.811+0000 7f17accef640 1 -- 172.21.15.45:0/523013849 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440e8e5340 con 0x56440e9e4cb0 2024-09-10T10:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.812+0000 7f17accef640 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e70a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f17940b8590 tx=0x7f1794031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ea33af5827d384 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.814+0000 7f179f7fe640 1 -- 172.21.15.45:0/523013849 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794049070 con 0x56440e9e4cb0 2024-09-10T10:08:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.814+0000 7f179f7fe640 1 -- 172.21.15.45:0/523013849 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940bc830 con 0x56440e9e4cb0 2024-09-10T10:08:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.814+0000 7f179f7fe640 1 -- 172.21.15.45:0/523013849 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940b87a0 con 0x56440e9e4cb0 2024-09-10T10:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.815+0000 7f17b25c8740 1 -- 172.21.15.45:0/523013849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 msgr2=0x56440e9e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.815+0000 7f17b25c8740 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e70a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f17940b8590 tx=0x7f1794031490 comp rx=0 tx=0).stop 2024-09-10T10:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.815+0000 7f17adcf1640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2e20 0x7f178c0a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f179800daf0 tx=0x7f1798008040 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.816+0000 7f17b25c8740 1 -- 172.21.15.45:0/523013849 shutdown_connections 2024-09-10T10:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.816+0000 7f17b25c8740 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e70a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.816+0000 7f17b25c8740 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.816+0000 7f17b25c8740 1 --2- 172.21.15.45:0/523013849 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9d9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.816+0000 7f17b25c8740 1 -- 172.21.15.45:0/523013849 >> 172.21.15.45:0/523013849 conn(0x56440e9d6060 msgr2=0x56440e9d6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.816+0000 7f17b25c8740 1 -- 172.21.15.45:0/523013849 shutdown_connections 2024-09-10T10:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.817+0000 7f17b25c8740 1 -- 172.21.15.45:0/523013849 wait complete. 2024-09-10T10:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.818+0000 7f17b25c8740 1 -- start start 2024-09-10T10:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.818+0000 7f17b25c8740 1 Processor -- start 2024-09-10T10:08:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.819+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9d9af0 0x56440e9e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.819+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9da410 0x56440e9e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.819+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.819+0000 7f17b25c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56440e9e7640 con 0x56440e9d9af0 2024-09-10T10:08:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.819+0000 7f17b25c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56440e8c7290 con 0x56440e9e4cb0 2024-09-10T10:08:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.819+0000 7f17b25c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56440e9e77c0 con 0x56440e9da410 2024-09-10T10:08:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.819+0000 7f17b25c8740 10 client.0 _ll_register_callbacks cb 0x56440e8f8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.820+0000 7f17ad4f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9d9af0 0x56440e9e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.820+0000 7f17ad4f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9d9af0 0x56440e9e1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35948/0 (socket says 172.21.15.45:35948) 2024-09-10T10:08:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.820+0000 7f17ad4f0640 1 -- 172.21.15.45:0/2331440446 learned_addr learned my addr 172.21.15.45:0/2331440446 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.820+0000 7f17accef640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9da410 0x56440e9e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.820+0000 7f17adcf1640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.821+0000 7f17ad4f0640 1 -- 172.21.15.45:0/2331440446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9da410 msgr2=0x56440e9e2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.821+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9da410 0x56440e9e2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.821+0000 7f17ad4f0640 1 -- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 msgr2=0x56440e9e2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.821+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9e2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.821+0000 7f17ad4f0640 1 -- 172.21.15.45:0/2331440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440e651210 con 0x56440e9d9af0 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.821+0000 7f17adcf1640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9e2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.822+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9d9af0 0x56440e9e1c10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f17a0003fc0 tx=0x7f17a00044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3feb28e3cfe742c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.822+0000 7f179cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:08:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.823+0000 7f179cff9640 1 -- 172.21.15.45:0/2331440446 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011900 con 0x56440e9d9af0 2024-09-10T10:08:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.823+0000 7f179cff9640 1 -- 172.21.15.45:0/2331440446 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f17a0011aa0 con 0x56440e9d9af0 2024-09-10T10:08:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.823+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17940bc830 con 0x56440e9d9af0 2024-09-10T10:08:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.824+0000 7f179cff9640 1 -- 172.21.15.45:0/2331440446 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f17a0011ca0 con 0x56440e9d9af0 2024-09-10T10:08:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.824+0000 7f17b25c8740 10 client.15429 fetch_fsmap learned FSMap version 19 2024-09-10T10:08:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.824+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56440e71d620 con 0x56440e9d9af0 2024-09-10T10:08:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.825+0000 7f179cff9640 1 -- 172.21.15.45:0/2331440446 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0011e40 con 0x56440e9d9af0 2024-09-10T10:08:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.825+0000 7f179cff9640 1 -- 172.21.15.45:0/2331440446 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f17a001b740 con 0x56440e9d9af0 2024-09-10T10:08:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.831+0000 7f17b25c8740 10 client.15429 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:08:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.831+0000 7f17b25c8740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.831+0000 7f17b25c8740 20 client.15429 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.831+0000 7f17b25c8740 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440e9f49c0 0x56440e9f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.831+0000 7f17b25c8740 4 client.15429 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:08:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.831+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f17940bc830 con 0x56440e9f49c0 2024-09-10T10:08:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.832+0000 7f17accef640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440e9f49c0 0x56440e9f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.834+0000 7f17accef640 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440e9f49c0 0x56440e9f6db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc801a2aaa9903 server_cookie=9d3959a24b747d4a in_seq=0 out_seq=0 2024-09-10T10:08:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.834+0000 7f179cff9640 10 client.15429 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.837+0000 7f179cff9640 1 -- 172.21.15.45:0/2331440446 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f17940bc830 con 0x56440e9f49c0 2024-09-10T10:08:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.837+0000 7f179cff9640 10 client.15429 handle_command_reply: tid=0 2024-09-10T10:08:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.837+0000 7f17b25c8740 1 client.15429 shutdown 2024-09-10T10:08:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.840+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9d9af0 msgr2=0x56440e9e1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.840+0000 7f17b25c8740 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9d9af0 0x56440e9e1c10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f17a0003fc0 tx=0x7f17a00044d0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.846+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 shutdown_connections 2024-09-10T10:08:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.846+0000 7f17b25c8740 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440e9f49c0 0x56440e9f6db0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.846+0000 7f17b25c8740 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9e2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.846+0000 7f17b25c8740 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9da410 0x56440e9e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.846+0000 7f17b25c8740 1 --2- 172.21.15.45:0/2331440446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9d9af0 0x56440e9e1c10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.846+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 >> 172.21.15.45:0/2331440446 conn(0x56440e9d6060 msgr2=0x56440e9dbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.846+0000 7f179cff9640 0 client.15429 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.848+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 shutdown_connections 2024-09-10T10:08:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.848+0000 7f17b25c8740 1 -- 172.21.15.45:0/2331440446 wait complete. 2024-09-10T10:08:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.848+0000 7f17b25c8740 20 client.15429 trim_cache size 0 max 16384 2024-09-10T10:08:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.886+0000 7f17b25c8740 1 Processor -- start 2024-09-10T10:08:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.886+0000 7f17b25c8740 1 -- start start 2024-09-10T10:08:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.887+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.887+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9da410 0x56440e9e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.887+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.887+0000 7f17b25c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81adb50 con 0x56440e9da410 2024-09-10T10:08:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.887+0000 7f17b25c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81b03f0 con 0x56440e9e4cb0 2024-09-10T10:08:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.887+0000 7f17b25c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81ae1e0 con 0x56440e9d9af0 2024-09-10T10:08:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.889+0000 7f17adcf1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.889+0000 7f17adcf1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:39238/0 (socket says 172.21.15.45:39238) 2024-09-10T10:08:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.889+0000 7f17adcf1640 1 -- 172.21.15.45:0/1814043946 learned_addr learned my addr 172.21.15.45:0/1814043946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.890+0000 7f17accef640 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9da410 0x56440e9e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.890+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.890+0000 7f17accef640 1 -- 172.21.15.45:0/1814043946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 msgr2=0x56440e9e24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.890+0000 7f17accef640 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9e24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.890+0000 7f17accef640 1 -- 172.21.15.45:0/1814043946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 msgr2=0x56440e9fc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.890+0000 7f17accef640 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9fc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.890+0000 7f17accef640 1 -- 172.21.15.45:0/1814043946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440e9d02f0 con 0x56440e9da410 2024-09-10T10:08:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.891+0000 7f17accef640 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9da410 0x56440e9e3b80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x56440e9dc690 tx=0x7f1794009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3de7a5b56effd3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.892+0000 7f179effd640 1 -- 172.21.15.45:0/1814043946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794049040 con 0x56440e9da410 2024-09-10T10:08:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.892+0000 7f179effd640 1 -- 172.21.15.45:0/1814043946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940c4da0 con 0x56440e9da410 2024-09-10T10:08:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.892+0000 7f179effd640 1 -- 172.21.15.45:0/1814043946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940c02d0 con 0x56440e9da410 2024-09-10T10:08:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.893+0000 7f17b25c8740 1 -- 172.21.15.45:0/1814043946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9da410 msgr2=0x56440e9e3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.893+0000 7f17b25c8740 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9da410 0x56440e9e3b80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x56440e9dc690 tx=0x7f1794009260 comp rx=0 tx=0).stop 2024-09-10T10:08:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.896+0000 7f17b25c8740 1 -- 172.21.15.45:0/1814043946 shutdown_connections 2024-09-10T10:08:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.896+0000 7f17b25c8740 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9e4cb0 0x56440e9fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.896+0000 7f17b25c8740 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9da410 0x56440e9e3b80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.896+0000 7f17b25c8740 1 --2- 172.21.15.45:0/1814043946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.896+0000 7f17b25c8740 1 -- 172.21.15.45:0/1814043946 >> 172.21.15.45:0/1814043946 conn(0x56440e9d6060 msgr2=0x56440e9e3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.900+0000 7f17b25c8740 1 -- 172.21.15.45:0/1814043946 shutdown_connections 2024-09-10T10:08:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.900+0000 7f17b25c8740 1 -- 172.21.15.45:0/1814043946 wait complete. 2024-09-10T10:08:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.901+0000 7f17b25c8740 1 -- start start 2024-09-10T10:08:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.901+0000 7f17b25c8740 1 Processor -- start 2024-09-10T10:08:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.902+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.902+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.902+0000 7f17b25c8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.902+0000 7f17b25c8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81ae8b0 con 0x56440e9e4cb0 2024-09-10T10:08:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.902+0000 7f17b25c8740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81afd20 con 0x56440e9da410 2024-09-10T10:08:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.902+0000 7f17b25c8740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f17a81aef80 con 0x56440e9d9af0 2024-09-10T10:08:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.902+0000 7f17b25c8740 10 client.0 _ll_register_callbacks cb 0x7f17a0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.905+0000 7f17ad4f0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.905+0000 7f17accef640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.906+0000 7f17adcf1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.906+0000 7f17adcf1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35960/0 (socket says 172.21.15.45:35960) 2024-09-10T10:08:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.906+0000 7f17adcf1640 1 -- 172.21.15.45:0/3682515591 learned_addr learned my addr 172.21.15.45:0/3682515591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.908+0000 7f17ad4f0640 1 -- 172.21.15.45:0/3682515591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 msgr2=0x56440e9e84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.908+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.908+0000 7f17ad4f0640 1 -- 172.21.15.45:0/3682515591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 msgr2=0x56440e9e8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.908+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.908+0000 7f17ad4f0640 1 -- 172.21.15.45:0/3682515591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440e9e9280 con 0x56440e9d9af0 2024-09-10T10:08:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.910+0000 7f17ad4f0640 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9ed1d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000f80 tx=0x7f17a00059f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dffff63abb05f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.911+0000 7f179cff9640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:08:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.914+0000 7f179cff9640 1 -- 172.21.15.45:0/3682515591 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a002b040 con 0x56440e9d9af0 2024-09-10T10:08:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.914+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17940c4da0 con 0x56440e9d9af0 2024-09-10T10:08:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.914+0000 7f179cff9640 1 -- 172.21.15.45:0/3682515591 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8512+0+0 (secure 0 0 0) 0x7f17a0017070 con 0x56440e9d9af0 2024-09-10T10:08:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.919+0000 7f179cff9640 1 -- 172.21.15.45:0/3682515591 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f17a0003d10 con 0x56440e9d9af0 2024-09-10T10:08:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.919+0000 7f17b25c8740 10 client.25042 fetch_fsmap learned FSMap version 19 2024-09-10T10:08:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.919+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x56440ea58660 con 0x56440e9d9af0 2024-09-10T10:08:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.927+0000 7f179cff9640 1 -- 172.21.15.45:0/3682515591 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940c4da0 con 0x56440e9d9af0 2024-09-10T10:08:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.927+0000 7f179cff9640 1 -- 172.21.15.45:0/3682515591 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 19) ==== 8330+0+0 (secure 0 0 0) 0x7f17a0037030 con 0x56440e9d9af0 2024-09-10T10:08:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.929+0000 7f17b25c8740 10 client.25042 fetch_fsmap finished waiting for FSMap version 19 2024-09-10T10:08:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.929+0000 7f17b25c8740 10 client.25042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.929+0000 7f17b25c8740 20 client.25042 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.929+0000 7f17b25c8740 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440ea58d80 0x56440ea5b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.929+0000 7f17b25c8740 4 client.25042 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:08:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.929+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56440e94ba20 con 0x56440ea58d80 2024-09-10T10:08:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.933+0000 7f17accef640 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440ea58d80 0x56440ea5b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.945+0000 7f17accef640 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440ea58d80 0x56440ea5b1a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5dae832d648bf2 server_cookie=da4f253a1af122d5 in_seq=0 out_seq=0 2024-09-10T10:08:08.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:08.947+0000 7f179cff9640 10 client.25042 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.653+0000 7f179cff9640 1 -- 172.21.15.45:0/3682515591 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56440e94ba20 con 0x56440ea58d80 2024-09-10T10:08:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.653+0000 7f179cff9640 10 client.25042 handle_command_reply: tid=0 2024-09-10T10:08:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.653+0000 7f17b25c8740 1 client.25042 shutdown 2024-09-10T10:08:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.653+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 msgr2=0x56440e9ed1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.653+0000 7f17b25c8740 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9ed1d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000f80 tx=0x7f17a00059f0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.654+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 shutdown_connections 2024-09-10T10:08:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.654+0000 7f17b25c8740 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56440ea58d80 0x56440ea5b1a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.654+0000 7f17b25c8740 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56440e9e4cb0 0x56440e9e8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.654+0000 7f17b25c8740 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56440e9da410 0x56440e9e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.654+0000 7f17b25c8740 1 --2- 172.21.15.45:0/3682515591 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56440e9d9af0 0x56440e9ed1d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.654+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 >> 172.21.15.45:0/3682515591 conn(0x56440e9d6060 msgr2=0x56440e9d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.654+0000 7f179cff9640 0 client.25042 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.656+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 shutdown_connections 2024-09-10T10:08:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.656+0000 7f17b25c8740 1 -- 172.21.15.45:0/3682515591 wait complete. 2024-09-10T10:08:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.656+0000 7f17b25c8740 20 client.25042 trim_cache size 0 max 16384 2024-09-10T10:08:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:08:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:08:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:08:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:08:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.659+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2e20 msgr2=0x7f178c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.659+0000 7f17af754640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2e20 0x7f178c0a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f179800daf0 tx=0x7f1798008040 comp rx=0 tx=0).stop 2024-09-10T10:08:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.659+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a8101370 msgr2=0x7f17a810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.659+0000 7f17af754640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a8101370 0x7f17a810e870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f179402a940 tx=0x7f1794003b90 comp rx=0 tx=0).stop 2024-09-10T10:08:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.660+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 shutdown_connections 2024-09-10T10:08:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.660+0000 7f17af754640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f17a81069e0 0x7f17a810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.660+0000 7f17af754640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17a8101370 0x7f17a810e870 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.660+0000 7f17af754640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f178c0a2e20 0x7f178c0a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.660+0000 7f17af754640 1 --2- 172.21.15.45:0/601656272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f17a81009d0 0x7f17a810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.660+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 >> 172.21.15.45:0/601656272 conn(0x7f17a80fc6b0 msgr2=0x7f17a80fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.661+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 shutdown_connections 2024-09-10T10:08:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:09.661+0000 7f17af754640 1 -- 172.21.15.45:0/601656272 wait complete. 2024-09-10T10:08:10.150 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:08:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i[64742]: starting mds.i at 2024-09-10T10:08:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:09 smithi162 ceph-mon[34963]: pgmap v151: 97 pgs: 97 active+clean; 40 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 820 MiB/s wr, 2.24k op/s 2024-09-10T10:08:10.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:09 smithi045 ceph-mon[31114]: pgmap v151: 97 pgs: 97 active+clean; 40 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 820 MiB/s wr, 2.24k op/s 2024-09-10T10:08:10.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:09 smithi047 ceph-mon[35244]: pgmap v151: 97 pgs: 97 active+clean; 40 GiB data, 221 GiB used, 852 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 820 MiB/s wr, 2.24k op/s 2024-09-10T10:08:11.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:10 smithi162 ceph-mon[34963]: pgmap v152: 97 pgs: 97 active+clean; 41 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 886 MiB/s wr, 2.59k op/s 2024-09-10T10:08:11.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:10 smithi162 ceph-mon[34963]: daemon mds.i restarted 2024-09-10T10:08:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:10 smithi045 ceph-mon[31114]: pgmap v152: 97 pgs: 97 active+clean; 41 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 886 MiB/s wr, 2.59k op/s 2024-09-10T10:08:11.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:10 smithi045 ceph-mon[31114]: daemon mds.i restarted 2024-09-10T10:08:11.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:10 smithi047 ceph-mon[35244]: pgmap v152: 97 pgs: 97 active+clean; 41 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 886 MiB/s wr, 2.59k op/s 2024-09-10T10:08:11.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:10 smithi047 ceph-mon[35244]: daemon mds.i restarted 2024-09-10T10:08:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:11 smithi162 ceph-mon[34963]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:08:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:11 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.162:6838/2699160150,v1:172.21.15.162:6839/2699160150] up:boot 2024-09-10T10:08:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:11 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:08:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:11 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:08:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:11 smithi162 ceph-mon[34963]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:08:12.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:11 smithi045 ceph-mon[31114]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:08:12.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:11 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.162:6838/2699160150,v1:172.21.15.162:6839/2699160150] up:boot 2024-09-10T10:08:12.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:11 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:08:12.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:11 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:08:12.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:11 smithi045 ceph-mon[31114]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:08:12.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:11 smithi047 ceph-mon[35244]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T10:08:12.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:11 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.162:6838/2699160150,v1:172.21.15.162:6839/2699160150] up:boot 2024-09-10T10:08:12.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:11 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T10:08:12.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:11 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:08:12.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:11 smithi047 ceph-mon[35244]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-10T10:08:13.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:12 smithi162 ceph-mon[34963]: pgmap v154: 97 pgs: 97 active+clean; 41 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 849 MiB/s wr, 2.66k op/s 2024-09-10T10:08:13.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:12 smithi045 ceph-mon[31114]: pgmap v154: 97 pgs: 97 active+clean; 41 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 849 MiB/s wr, 2.66k op/s 2024-09-10T10:08:13.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:12 smithi047 ceph-mon[35244]: pgmap v154: 97 pgs: 97 active+clean; 41 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 849 MiB/s wr, 2.66k op/s 2024-09-10T10:08:15.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:15 smithi162 ceph-mon[34963]: pgmap v155: 97 pgs: 97 active+clean; 43 GiB data, 233 GiB used, 840 GiB / 1.0 TiB avail; 823 KiB/s rd, 778 MiB/s wr, 3.59k op/s 2024-09-10T10:08:15.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:15 smithi047 ceph-mon[35244]: pgmap v155: 97 pgs: 97 active+clean; 43 GiB data, 233 GiB used, 840 GiB / 1.0 TiB avail; 823 KiB/s rd, 778 MiB/s wr, 3.59k op/s 2024-09-10T10:08:15.410 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:15 smithi045 ceph-mon[31114]: pgmap v155: 97 pgs: 97 active+clean; 43 GiB data, 233 GiB used, 840 GiB / 1.0 TiB avail; 823 KiB/s rd, 778 MiB/s wr, 3.59k op/s 2024-09-10T10:08:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:17 smithi162 ceph-mon[34963]: pgmap v156: 97 pgs: 97 active+clean; 44 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 MiB/s wr, 4.13k op/s 2024-09-10T10:08:17.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:17 smithi045 ceph-mon[31114]: pgmap v156: 97 pgs: 97 active+clean; 44 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 MiB/s wr, 4.13k op/s 2024-09-10T10:08:17.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:17 smithi047 ceph-mon[35244]: pgmap v156: 97 pgs: 97 active+clean; 44 GiB data, 235 GiB used, 838 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 MiB/s wr, 4.13k op/s 2024-09-10T10:08:19.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:19 smithi162 ceph-mon[34963]: pgmap v157: 97 pgs: 97 active+clean; 44 GiB data, 238 GiB used, 835 GiB / 1.0 TiB avail; 834 KiB/s rd, 808 MiB/s wr, 4.21k op/s 2024-09-10T10:08:19.406 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:19 smithi045 ceph-mon[31114]: pgmap v157: 97 pgs: 97 active+clean; 44 GiB data, 238 GiB used, 835 GiB / 1.0 TiB avail; 834 KiB/s rd, 808 MiB/s wr, 4.21k op/s 2024-09-10T10:08:19.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:19 smithi047 ceph-mon[35244]: pgmap v157: 97 pgs: 97 active+clean; 44 GiB data, 238 GiB used, 835 GiB / 1.0 TiB avail; 834 KiB/s rd, 808 MiB/s wr, 4.21k op/s 2024-09-10T10:08:21.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:21 smithi162 ceph-mon[34963]: pgmap v158: 97 pgs: 97 active+clean; 46 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 827 MiB/s wr, 4.08k op/s 2024-09-10T10:08:21.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:21 smithi045 ceph-mon[31114]: pgmap v158: 97 pgs: 97 active+clean; 46 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 827 MiB/s wr, 4.08k op/s 2024-09-10T10:08:21.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:21 smithi047 ceph-mon[35244]: pgmap v158: 97 pgs: 97 active+clean; 46 GiB data, 244 GiB used, 829 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 827 MiB/s wr, 4.08k op/s 2024-09-10T10:08:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.020+0000 7ffa85227640 1 -- 172.21.15.45:0/3136262551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa80108d10 msgr2=0x7ffa8010cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.020+0000 7ffa85227640 1 --2- 172.21.15.45:0/3136262551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa80108d10 0x7ffa8010cd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009900 tx=0x7ffa7402a410 comp rx=0 tx=0).stop 2024-09-10T10:08:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.021+0000 7ffa85227640 1 -- 172.21.15.45:0/3136262551 shutdown_connections 2024-09-10T10:08:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.021+0000 7ffa85227640 1 --2- 172.21.15.45:0/3136262551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa8010d4b0 0x7ffa80115000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.021+0000 7ffa85227640 1 --2- 172.21.15.45:0/3136262551 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa80108d10 0x7ffa8010cd50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.021+0000 7ffa85227640 1 --2- 172.21.15.45:0/3136262551 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa80108360 0x7ffa80108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.021+0000 7ffa85227640 1 -- 172.21.15.45:0/3136262551 >> 172.21.15.45:0/3136262551 conn(0x7ffa80077ad0 msgr2=0x7ffa80079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.023+0000 7ffa85227640 1 -- 172.21.15.45:0/3136262551 shutdown_connections 2024-09-10T10:08:23.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.023+0000 7ffa85227640 1 -- 172.21.15.45:0/3136262551 wait complete. 2024-09-10T10:08:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.025+0000 7ffa85227640 1 Processor -- start 2024-09-10T10:08:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.025+0000 7ffa85227640 1 -- start start 2024-09-10T10:08:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa85227640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa80108360 0x7ffa80082980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa85227640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa8010d4b0 0x7ffa8007ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa85227640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa8007cf80 0x7ffa8007d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa85227640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8007b490 con 0x7ffa8007cf80 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa85227640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8007b310 con 0x7ffa80108360 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa85227640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8007b610 con 0x7ffa8010d4b0 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa7f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa8007cf80 0x7ffa8007d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa7f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa8007cf80 0x7ffa8007d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41148/0 (socket says 172.21.15.45:41148) 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.026+0000 7ffa7f59e640 1 -- 172.21.15.45:0/2152835177 learned_addr learned my addr 172.21.15.45:0/2152835177 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.027+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa80108360 0x7ffa80082980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.027+0000 7ffa7f59e640 1 -- 172.21.15.45:0/2152835177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa8010d4b0 msgr2=0x7ffa8007ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.027+0000 7ffa7f59e640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa8010d4b0 0x7ffa8007ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.027+0000 7ffa7f59e640 1 -- 172.21.15.45:0/2152835177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa80108360 msgr2=0x7ffa80082980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.027+0000 7ffa7f59e640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa80108360 0x7ffa80082980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.027+0000 7ffa7f59e640 1 -- 172.21.15.45:0/2152835177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa801b38d0 con 0x7ffa8007cf80 2024-09-10T10:08:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.027+0000 7ffa7f59e640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa8007cf80 0x7ffa8007d430 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800da70 tx=0x7ffa7800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.028+0000 7ffa67fff640 1 -- 172.21.15.45:0/2152835177 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78017070 con 0x7ffa8007cf80 2024-09-10T10:08:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.028+0000 7ffa67fff640 1 -- 172.21.15.45:0/2152835177 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa78003e50 con 0x7ffa8007cf80 2024-09-10T10:08:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.028+0000 7ffa67fff640 1 -- 172.21.15.45:0/2152835177 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78010460 con 0x7ffa8007cf80 2024-09-10T10:08:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.029+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa801b3bc0 con 0x7ffa8007cf80 2024-09-10T10:08:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.029+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa801b4090 con 0x7ffa8007cf80 2024-09-10T10:08:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.033+0000 7ffa87fc3740 1 Processor -- start 2024-09-10T10:08:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa67fff640 1 -- 172.21.15.45:0/2152835177 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa78004000 con 0x7ffa8007cf80 2024-09-10T10:08:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa87fc3740 1 -- start start 2024-09-10T10:08:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25daf0 0x55d8eb25ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 0x55d8eb268770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb268cb0 0x55d8eb26b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d8eb1c22f0 con 0x55d8eb268cb0 2024-09-10T10:08:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d8eb16a270 con 0x55d8eb25daf0 2024-09-10T10:08:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.036+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d8eb1be2b0 con 0x55d8eb25e410 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.037+0000 7ffa7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb268cb0 0x55d8eb26b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.037+0000 7ffa7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb268cb0 0x55d8eb26b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41164/0 (socket says 172.21.15.45:41164) 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.037+0000 7ffa7e59c640 1 -- 172.21.15.45:0/2276225416 learned_addr learned my addr 172.21.15.45:0/2276225416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.037+0000 7ffa67fff640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ffa680a2f50 0x7ffa680a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa7e59c640 1 -- 172.21.15.45:0/2276225416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 msgr2=0x55d8eb268770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/2276225416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 0x55d8eb268770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa7e59c640 1 -- 172.21.15.45:0/2276225416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25daf0 msgr2=0x55d8eb25ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/2276225416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25daf0 0x55d8eb25ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa7e59c640 1 -- 172.21.15.45:0/2276225416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8eb169340 con 0x55d8eb268cb0 2024-09-10T10:08:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa67fff640 1 -- 172.21.15.45:0/2152835177 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7ffa780ceff0 con 0x7ffa8007cf80 2024-09-10T10:08:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/2276225416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb268cb0 0x55d8eb26b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009230 tx=0x7ffa740314d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e22a9e2d302af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.038+0000 7ffa7f59e640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ffa680a2f50 0x7ffa680a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.039+0000 7ffa657fa640 1 -- 172.21.15.45:0/2276225416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7403e030 con 0x55d8eb268cb0 2024-09-10T10:08:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.039+0000 7ffa657fa640 1 -- 172.21.15.45:0/2276225416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa740035f0 con 0x55d8eb268cb0 2024-09-10T10:08:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.039+0000 7ffa657fa640 1 -- 172.21.15.45:0/2276225416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74035040 con 0x55d8eb268cb0 2024-09-10T10:08:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.040+0000 7ffa7f59e640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ffa680a2f50 0x7ffa680a5410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffa7809b280 tx=0x7ffa78004a40 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.040+0000 7ffa87fc3740 1 -- 172.21.15.45:0/2276225416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb268cb0 msgr2=0x55d8eb26b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.040+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/2276225416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb268cb0 0x55d8eb26b0a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009230 tx=0x7ffa740314d0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.041+0000 7ffa87fc3740 1 -- 172.21.15.45:0/2276225416 shutdown_connections 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.041+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/2276225416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb268cb0 0x55d8eb26b0a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.041+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/2276225416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 0x55d8eb268770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.041+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/2276225416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25daf0 0x55d8eb25ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.041+0000 7ffa87fc3740 1 -- 172.21.15.45:0/2276225416 >> 172.21.15.45:0/2276225416 conn(0x55d8eb25a060 msgr2=0x55d8eb25a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.041+0000 7ffa87fc3740 1 -- 172.21.15.45:0/2276225416 shutdown_connections 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.041+0000 7ffa87fc3740 1 -- 172.21.15.45:0/2276225416 wait complete. 2024-09-10T10:08:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.042+0000 7ffa87fc3740 1 -- start start 2024-09-10T10:08:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.042+0000 7ffa87fc3740 1 Processor -- start 2024-09-10T10:08:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.043+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25daf0 0x55d8eb265c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.043+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25e410 0x55d8eb266180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.043+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.044+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d8eb26b880 con 0x55d8eb25e410 2024-09-10T10:08:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.044+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d8eb14b290 con 0x55d8eb268cb0 2024-09-10T10:08:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.044+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d8eb26ba00 con 0x55d8eb25daf0 2024-09-10T10:08:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.044+0000 7ffa7e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.044+0000 7ffa7e59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2666c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:55904/0 (socket says 172.21.15.45:55904) 2024-09-10T10:08:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.044+0000 7ffa7e59c640 1 -- 172.21.15.45:0/717244292 learned_addr learned my addr 172.21.15.45:0/717244292 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.044+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25e410 0x55d8eb266180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.045+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25daf0 0x55d8eb265c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.046+0000 7ffa7ed9d640 1 -- 172.21.15.45:0/717244292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25daf0 msgr2=0x55d8eb265c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.046+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25daf0 0x55d8eb265c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.046+0000 7ffa7ed9d640 1 -- 172.21.15.45:0/717244292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 msgr2=0x55d8eb2666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.046+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.046+0000 7ffa7ed9d640 1 -- 172.21.15.45:0/717244292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8eaed5210 con 0x55d8eb25e410 2024-09-10T10:08:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.046+0000 7ffa87fc3740 10 client.0 _ll_register_callbacks cb 0x55d8eb17c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.046+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa740035f0 con 0x55d8eb25e410 2024-09-10T10:08:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.048+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25e410 0x55d8eb266180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000bf30 tx=0x7ffa7000d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd50916b73edbbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.048+0000 7ffa62ff5640 10 client.15453 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:08:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.048+0000 7ffa62ff5640 1 -- 172.21.15.45:0/717244292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7001e450 con 0x55d8eb25e410 2024-09-10T10:08:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.048+0000 7ffa62ff5640 1 -- 172.21.15.45:0/717244292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7ffa7001e5f0 con 0x55d8eb25e410 2024-09-10T10:08:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.049+0000 7ffa62ff5640 1 -- 172.21.15.45:0/717244292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ffa7001e850 con 0x55d8eb25e410 2024-09-10T10:08:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.049+0000 7ffa87fc3740 10 client.15453 fetch_fsmap learned FSMap version 21 2024-09-10T10:08:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.049+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=21}) -- 0x55d8eafa1620 con 0x55d8eb25e410 2024-09-10T10:08:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.049+0000 7ffa62ff5640 1 -- 172.21.15.45:0/717244292 <== mon.0 v2:172.21.15.45:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ffa70013070 con 0x55d8eb25e410 2024-09-10T10:08:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.049+0000 7ffa87fc3740 10 client.15453 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:08:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.050+0000 7ffa87fc3740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.050+0000 7ffa87fc3740 20 client.15453 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.050+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb278b10 0x55d8eb27afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.050+0000 7ffa87fc3740 4 client.15453 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:08:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.050+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8eb1cfa20 con 0x55d8eb278b10 2024-09-10T10:08:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.050+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb278b10 0x55d8eb27afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.052+0000 7ffa62ff5640 10 client.15453 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.052+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb278b10 0x55d8eb27afa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df590b4da089a423 server_cookie=cf765cb189a9051a in_seq=0 out_seq=0 2024-09-10T10:08:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.056+0000 7ffa62ff5640 1 -- 172.21.15.45:0/717244292 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8eb1cfa20 con 0x55d8eb278b10 2024-09-10T10:08:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.056+0000 7ffa62ff5640 10 client.15453 handle_command_reply: tid=0 2024-09-10T10:08:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.056+0000 7ffa87fc3740 1 client.15453 shutdown 2024-09-10T10:08:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.057+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25e410 msgr2=0x55d8eb266180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.057+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25e410 0x55d8eb266180 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000bf30 tx=0x7ffa7000d5f0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.057+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 shutdown_connections 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.058+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb278b10 0x55d8eb27afa0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.058+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.058+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25e410 0x55d8eb266180 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.058+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/717244292 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25daf0 0x55d8eb265c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.058+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 >> 172.21.15.45:0/717244292 conn(0x55d8eb25a060 msgr2=0x55d8eb25fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.059+0000 7ffa62ff5640 0 client.15453 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.059+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 shutdown_connections 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.059+0000 7ffa87fc3740 1 -- 172.21.15.45:0/717244292 wait complete. 2024-09-10T10:08:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.059+0000 7ffa87fc3740 20 client.15453 trim_cache size 0 max 16384 2024-09-10T10:08:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.095+0000 7ffa87fc3740 1 Processor -- start 2024-09-10T10:08:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.096+0000 7ffa87fc3740 1 -- start start 2024-09-10T10:08:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.096+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb267f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.096+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 0x55d8eb2664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.096+0000 7ffa7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb267f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb267f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41174/0 (socket says 172.21.15.45:41174) 2024-09-10T10:08:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa7e59c640 1 -- 172.21.15.45:0/1178454410 learned_addr learned my addr 172.21.15.45:0/1178454410 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801ba510 con 0x55d8eb25daf0 2024-09-10T10:08:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801bcdb0 con 0x55d8eb268cb0 2024-09-10T10:08:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801baba0 con 0x55d8eb25e410 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa7e59c640 1 -- 172.21.15.45:0/1178454410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 msgr2=0x55d8eb2664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 0x55d8eb2664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa7e59c640 1 -- 172.21.15.45:0/1178454410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 msgr2=0x55d8eb2808a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2808a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.097+0000 7ffa7e59c640 1 -- 172.21.15.45:0/1178454410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8eb259d30 con 0x55d8eb25daf0 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.098+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb267f70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa74003f40 tx=0x7ffa74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169da1b3b35032e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.098+0000 7ffa64ff9640 1 -- 172.21.15.45:0/1178454410 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74052070 con 0x55d8eb25daf0 2024-09-10T10:08:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.098+0000 7ffa64ff9640 1 -- 172.21.15.45:0/1178454410 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74005850 con 0x55d8eb25daf0 2024-09-10T10:08:23.101 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.098+0000 7ffa64ff9640 1 -- 172.21.15.45:0/1178454410 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74035040 con 0x55d8eb25daf0 2024-09-10T10:08:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.100+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 msgr2=0x55d8eb267f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:23.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.100+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb267f70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa74003f40 tx=0x7ffa74009260 comp rx=0 tx=0).stop 2024-09-10T10:08:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.102+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 shutdown_connections 2024-09-10T10:08:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.102+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb268cb0 0x55d8eb2808a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.102+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb25e410 0x55d8eb2664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.102+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1178454410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb267f70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.102+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 >> 172.21.15.45:0/1178454410 conn(0x55d8eb25a060 msgr2=0x55d8eb2591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.113+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 shutdown_connections 2024-09-10T10:08:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.113+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1178454410 wait complete. 2024-09-10T10:08:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.114+0000 7ffa87fc3740 1 -- start start 2024-09-10T10:08:23.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.114+0000 7ffa87fc3740 1 Processor -- start 2024-09-10T10:08:23.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.115+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb270cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.116+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25e410 0x55d8eb271230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.116+0000 7ffa87fc3740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb268cb0 0x55d8eb26dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.116+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801bb270 con 0x55d8eb25daf0 2024-09-10T10:08:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.116+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801bc6e0 con 0x55d8eb25e410 2024-09-10T10:08:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.116+0000 7ffa87fc3740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ffa801bb940 con 0x55d8eb268cb0 2024-09-10T10:08:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.116+0000 7ffa7ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25e410 0x55d8eb271230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa87fc3740 10 client.0 _ll_register_callbacks cb 0x7ffa7000b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7ed9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25e410 0x55d8eb271230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:55928/0 (socket says 172.21.15.45:55928) 2024-09-10T10:08:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7ed9d640 1 -- 172.21.15.45:0/1297412062 learned_addr learned my addr 172.21.15.45:0/1297412062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb268cb0 0x55d8eb26dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb270cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7ed9d640 1 -- 172.21.15.45:0/1297412062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb268cb0 msgr2=0x55d8eb26dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb268cb0 0x55d8eb26dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7ed9d640 1 -- 172.21.15.45:0/1297412062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 msgr2=0x55d8eb270cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb270cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.117+0000 7ffa7ed9d640 1 -- 172.21.15.45:0/1297412062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8eb26e200 con 0x55d8eb25e410 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.118+0000 7ffa7ed9d640 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25e410 0x55d8eb271230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa70012400 tx=0x7ffa700046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b0ac7bcb592113 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.118+0000 7ffa62ff5640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.119+0000 7ffa62ff5640 1 -- 172.21.15.45:0/1297412062 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa70004020 con 0x55d8eb25e410 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.119+0000 7ffa62ff5640 1 -- 172.21.15.45:0/1297412062 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7ffa700041c0 con 0x55d8eb25e410 2024-09-10T10:08:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.119+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa74005850 con 0x55d8eb25e410 2024-09-10T10:08:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.120+0000 7ffa62ff5640 1 -- 172.21.15.45:0/1297412062 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ffa700043c0 con 0x55d8eb25e410 2024-09-10T10:08:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.120+0000 7ffa87fc3740 10 client.25139 fetch_fsmap learned FSMap version 21 2024-09-10T10:08:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.120+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d8eb2e3f20 con 0x55d8eb25e410 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa62ff5640 1 -- 172.21.15.45:0/1297412062 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa74005850 con 0x55d8eb25e410 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa62ff5640 1 -- 172.21.15.45:0/1297412062 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ffa70003690 con 0x55d8eb25e410 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa87fc3740 10 client.25139 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa87fc3740 10 client.25139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa87fc3740 20 client.25139 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb2e4690 0x55d8eb2e6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa87fc3740 4 client.25139 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:08:23.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.121+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8eb1c9d70 con 0x55d8eb2e4690 2024-09-10T10:08:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.124+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb2e4690 0x55d8eb2e6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.135+0000 7ffa7e59c640 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb2e4690 0x55d8eb2e6a80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404a851eb7aa5edc server_cookie=b4d72d966289f7b1 in_seq=0 out_seq=0 2024-09-10T10:08:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:23.135+0000 7ffa62ff5640 10 client.25139 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:23 smithi162 ceph-mon[34963]: pgmap v159: 97 pgs: 97 active+clean; 46 GiB data, 244 GiB used, 828 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 746 MiB/s wr, 3.43k op/s 2024-09-10T10:08:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:23 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:08:23.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:23 smithi045 ceph-mon[31114]: pgmap v159: 97 pgs: 97 active+clean; 46 GiB data, 244 GiB used, 828 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 746 MiB/s wr, 3.43k op/s 2024-09-10T10:08:23.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:23 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:08:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:23 smithi047 ceph-mon[35244]: pgmap v159: 97 pgs: 97 active+clean; 46 GiB data, 244 GiB used, 828 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 746 MiB/s wr, 3.43k op/s 2024-09-10T10:08:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:23 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:08:25.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:25 smithi162 ceph-mon[34963]: pgmap v160: 97 pgs: 97 active+clean; 47 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 927 MiB/s wr, 3.50k op/s 2024-09-10T10:08:25.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:25 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:08:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:25 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:08:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:25 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:08:25.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:25 smithi045 ceph-mon[31114]: pgmap v160: 97 pgs: 97 active+clean; 47 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 927 MiB/s wr, 3.50k op/s 2024-09-10T10:08:25.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:25 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:08:25.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:25 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:08:25.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:25 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:08:25.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:25 smithi047 ceph-mon[35244]: pgmap v160: 97 pgs: 97 active+clean; 47 GiB data, 251 GiB used, 822 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 927 MiB/s wr, 3.50k op/s 2024-09-10T10:08:25.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:25 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:08:25.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:25 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:08:25.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:25 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:08:27.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:27 smithi162 ceph-mon[34963]: pgmap v161: 97 pgs: 97 active+clean; 48 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 812 MiB/s wr, 2.23k op/s 2024-09-10T10:08:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:27 smithi045 ceph-mon[31114]: pgmap v161: 97 pgs: 97 active+clean; 48 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 812 MiB/s wr, 2.23k op/s 2024-09-10T10:08:27.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:27 smithi047 ceph-mon[35244]: pgmap v161: 97 pgs: 97 active+clean; 48 GiB data, 252 GiB used, 821 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 812 MiB/s wr, 2.23k op/s 2024-09-10T10:08:29.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:29 smithi162 ceph-mon[34963]: pgmap v162: 97 pgs: 97 active+clean; 49 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 827 MiB/s wr, 1.84k op/s 2024-09-10T10:08:29.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:29 smithi045 ceph-mon[31114]: pgmap v162: 97 pgs: 97 active+clean; 49 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 827 MiB/s wr, 1.84k op/s 2024-09-10T10:08:29.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:29 smithi047 ceph-mon[35244]: pgmap v162: 97 pgs: 97 active+clean; 49 GiB data, 256 GiB used, 817 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 827 MiB/s wr, 1.84k op/s 2024-09-10T10:08:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa62ff5640 1 -- 172.21.15.45:0/1297412062 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8eb1c9d70 con 0x55d8eb2e4690 2024-09-10T10:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa62ff5640 10 client.25139 handle_command_reply: tid=0 2024-09-10T10:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 client.25139 shutdown 2024-09-10T10:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25e410 msgr2=0x55d8eb271230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25e410 0x55d8eb271230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffa70012400 tx=0x7ffa700046e0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 shutdown_connections 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d8eb2e4690 0x55d8eb2e6a80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d8eb268cb0 0x55d8eb26dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d8eb25e410 0x55d8eb271230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 --2- 172.21.15.45:0/1297412062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d8eb25daf0 0x55d8eb270cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.020+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 >> 172.21.15.45:0/1297412062 conn(0x55d8eb25a060 msgr2=0x55d8eb273b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.021+0000 7ffa62ff5640 0 client.25139 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.021+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 shutdown_connections 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.021+0000 7ffa87fc3740 1 -- 172.21.15.45:0/1297412062 wait complete. 2024-09-10T10:08:30.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.021+0000 7ffa87fc3740 20 client.25139 trim_cache size 0 max 16384 2024-09-10T10:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:08:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:08:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ffa680a2f50 msgr2=0x7ffa680a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ffa680a2f50 0x7ffa680a5410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffa7809b280 tx=0x7ffa78004a40 comp rx=0 tx=0).stop 2024-09-10T10:08:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa8007cf80 msgr2=0x7ffa8007d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa8007cf80 0x7ffa8007d430 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800da70 tx=0x7ffa7800df30 comp rx=0 tx=0).stop 2024-09-10T10:08:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 shutdown_connections 2024-09-10T10:08:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffa8007cf80 0x7ffa8007d430 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ffa8010d4b0 0x7ffa8007ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ffa680a2f50 0x7ffa680a5410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 --2- 172.21.15.45:0/2152835177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa80108360 0x7ffa80082980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 >> 172.21.15.45:0/2152835177 conn(0x7ffa80077ad0 msgr2=0x7ffa80079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.024+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 shutdown_connections 2024-09-10T10:08:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:30.025+0000 7ffa85227640 1 -- 172.21.15.45:0/2152835177 wait complete. 2024-09-10T10:08:31.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:31 smithi162 ceph-mon[34963]: pgmap v163: 97 pgs: 97 active+clean; 50 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 5.7 MiB/s rd, 908 MiB/s wr, 1.74k op/s 2024-09-10T10:08:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:31 smithi045 ceph-mon[31114]: pgmap v163: 97 pgs: 97 active+clean; 50 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 5.7 MiB/s rd, 908 MiB/s wr, 1.74k op/s 2024-09-10T10:08:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:31 smithi047 ceph-mon[35244]: pgmap v163: 97 pgs: 97 active+clean; 50 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 5.7 MiB/s rd, 908 MiB/s wr, 1.74k op/s 2024-09-10T10:08:33.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:33 smithi047 ceph-mon[35244]: pgmap v164: 97 pgs: 97 active+clean; 50 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 746 MiB/s wr, 1.17k op/s 2024-09-10T10:08:33.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:33 smithi162 ceph-mon[34963]: pgmap v164: 97 pgs: 97 active+clean; 50 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 746 MiB/s wr, 1.17k op/s 2024-09-10T10:08:33.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:33 smithi045 ceph-mon[31114]: pgmap v164: 97 pgs: 97 active+clean; 50 GiB data, 260 GiB used, 813 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 746 MiB/s wr, 1.17k op/s 2024-09-10T10:08:35.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:35 smithi162 ceph-mon[34963]: pgmap v165: 97 pgs: 97 active+clean; 50 GiB data, 264 GiB used, 808 GiB / 1.0 TiB avail; 4.8 MiB/s rd, 963 MiB/s wr, 1.52k op/s 2024-09-10T10:08:35.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:35 smithi045 ceph-mon[31114]: pgmap v165: 97 pgs: 97 active+clean; 50 GiB data, 264 GiB used, 808 GiB / 1.0 TiB avail; 4.8 MiB/s rd, 963 MiB/s wr, 1.52k op/s 2024-09-10T10:08:35.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:35 smithi047 ceph-mon[35244]: pgmap v165: 97 pgs: 97 active+clean; 50 GiB data, 264 GiB used, 808 GiB / 1.0 TiB avail; 4.8 MiB/s rd, 963 MiB/s wr, 1.52k op/s 2024-09-10T10:08:37.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:37 smithi045 ceph-mon[31114]: pgmap v166: 97 pgs: 97 active+clean; 50 GiB data, 264 GiB used, 808 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 808 MiB/s wr, 1.32k op/s 2024-09-10T10:08:37.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:37 smithi047 ceph-mon[35244]: pgmap v166: 97 pgs: 97 active+clean; 50 GiB data, 264 GiB used, 808 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 808 MiB/s wr, 1.32k op/s 2024-09-10T10:08:38.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:37 smithi162 ceph-mon[34963]: pgmap v166: 97 pgs: 97 active+clean; 50 GiB data, 264 GiB used, 808 GiB / 1.0 TiB avail; 3.1 MiB/s rd, 808 MiB/s wr, 1.32k op/s 2024-09-10T10:08:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:39 smithi162 ceph-mon[34963]: pgmap v167: 97 pgs: 97 active+clean; 50 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 855 MiB/s wr, 1.55k op/s 2024-09-10T10:08:40.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:39 smithi045 ceph-mon[31114]: pgmap v167: 97 pgs: 97 active+clean; 50 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 855 MiB/s wr, 1.55k op/s 2024-09-10T10:08:40.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:39 smithi047 ceph-mon[35244]: pgmap v167: 97 pgs: 97 active+clean; 50 GiB data, 266 GiB used, 807 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 855 MiB/s wr, 1.55k op/s 2024-09-10T10:08:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:41 smithi162 ceph-mon[34963]: pgmap v168: 97 pgs: 97 active+clean; 50 GiB data, 269 GiB used, 803 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 893 MiB/s wr, 1.77k op/s 2024-09-10T10:08:42.154 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:41 smithi045 ceph-mon[31114]: pgmap v168: 97 pgs: 97 active+clean; 50 GiB data, 269 GiB used, 803 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 893 MiB/s wr, 1.77k op/s 2024-09-10T10:08:42.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:41 smithi047 ceph-mon[35244]: pgmap v168: 97 pgs: 97 active+clean; 50 GiB data, 269 GiB used, 803 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 893 MiB/s wr, 1.77k op/s 2024-09-10T10:08:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.232+0000 7f8fec42c640 1 -- 172.21.15.45:0/156888785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073790 msgr2=0x7f8fe410ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.232+0000 7f8fec42c640 1 --2- 172.21.15.45:0/156888785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073790 0x7f8fe410ead0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4013f10 tx=0x7f8fd40329c0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.232+0000 7f8fec42c640 1 -- 172.21.15.45:0/156888785 shutdown_connections 2024-09-10T10:08:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.233+0000 7f8fec42c640 1 --2- 172.21.15.45:0/156888785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fe41091f0 0x7f8fe4109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.233+0000 7f8fec42c640 1 --2- 172.21.15.45:0/156888785 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073c00 0x7f8fe410f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.233+0000 7f8fec42c640 1 --2- 172.21.15.45:0/156888785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073790 0x7f8fe410ead0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.233+0000 7f8fec42c640 1 -- 172.21.15.45:0/156888785 >> 172.21.15.45:0/156888785 conn(0x7f8fe406d210 msgr2=0x7f8fe406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.233+0000 7f8fec42c640 1 -- 172.21.15.45:0/156888785 shutdown_connections 2024-09-10T10:08:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.233+0000 7f8fec42c640 1 -- 172.21.15.45:0/156888785 wait complete. 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 Processor -- start 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 -- start start 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073790 0x7f8fe41a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073c00 0x7f8fe41a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fe41091f0 0x7f8fe419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4116650 con 0x7f8fe4073c00 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe41164d0 con 0x7f8fe41091f0 2024-09-10T10:08:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.234+0000 7f8fec42c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe41167d0 con 0x7f8fe4073790 2024-09-10T10:08:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073790 0x7f8fe41a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073790 0x7f8fe41a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:49676/0 (socket says 172.21.15.45:49676) 2024-09-10T10:08:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 -- 172.21.15.45:0/2492553542 learned_addr learned my addr 172.21.15.45:0/2492553542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073c00 0x7f8fe41a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 -- 172.21.15.45:0/2492553542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fe41091f0 msgr2=0x7f8fe419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fe41091f0 0x7f8fe419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 -- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073c00 msgr2=0x7f8fe41a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073c00 0x7f8fe41a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.235+0000 7f8fea1c8640 1 -- 172.21.15.45:0/2492553542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe419f010 con 0x7f8fe4073790 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.236+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073c00 0x7f8fe41a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.236+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073790 0x7f8fe41a46e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4006010 tx=0x7f8fd4004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.236+0000 7f8fdb7fe640 1 -- 172.21.15.45:0/2492553542 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4004030 con 0x7f8fe4073790 2024-09-10T10:08:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.236+0000 7f8fdb7fe640 1 -- 172.21.15.45:0/2492553542 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd40041d0 con 0x7f8fe4073790 2024-09-10T10:08:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.236+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe419f2a0 con 0x7f8fe4073790 2024-09-10T10:08:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.236+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe41ab5a0 con 0x7f8fe4073790 2024-09-10T10:08:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.237+0000 7f8fdb7fe640 1 -- 172.21.15.45:0/2492553542 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40132b0 con 0x7f8fe4073790 2024-09-10T10:08:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.240+0000 7f8fdb7fe640 1 -- 172.21.15.45:0/2492553542 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8fd403e070 con 0x7f8fe4073790 2024-09-10T10:08:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.241+0000 7f8fdb7fe640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8fd00a2e20 0x7f8fd00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.242+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8fd00a2e20 0x7f8fd00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.242+0000 7f8fdb7fe640 1 -- 172.21.15.45:0/2492553542 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f8fd40f8d20 con 0x7f8fe4073790 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.242+0000 7f8fef1bf740 1 Processor -- start 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.242+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8fd00a2e20 0x7f8fd00a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fe406d9e0 tx=0x7f8fe0009110 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.242+0000 7f8fef1bf740 1 -- start start 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c638ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c639410 0x56389c643770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c6460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56389c59d2f0 con 0x56389c643cb0 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56389c545270 con 0x56389c638af0 2024-09-10T10:08:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56389c5992b0 con 0x56389c639410 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c6460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c6460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55062/0 (socket says 172.21.15.45:55062) 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 -- 172.21.15.45:0/3548040851 learned_addr learned my addr 172.21.15.45:0/3548040851 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c639410 0x56389c643770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 -- 172.21.15.45:0/3548040851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c639410 msgr2=0x56389c643770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c639410 0x56389c643770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 -- 172.21.15.45:0/3548040851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 msgr2=0x56389c638ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c638ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.243+0000 7f8fea9c9640 1 -- 172.21.15.45:0/3548040851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389c544340 con 0x56389c643cb0 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.244+0000 7f8fea9c9640 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c6460a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009d20 tx=0x7f8fdc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6b487f891d96b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.244+0000 7f8fd8bfe640 1 -- 172.21.15.45:0/3548040851 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00bd40 con 0x56389c643cb0 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.244+0000 7f8fd8bfe640 1 -- 172.21.15.45:0/3548040851 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc017070 con 0x56389c643cb0 2024-09-10T10:08:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.244+0000 7f8fd8bfe640 1 -- 172.21.15.45:0/3548040851 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc011790 con 0x56389c643cb0 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3548040851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 msgr2=0x56389c6460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c6460a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009d20 tx=0x7f8fdc00b420 comp rx=0 tx=0).stop 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3548040851 shutdown_connections 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c6460a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c639410 0x56389c643770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3548040851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c638ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3548040851 >> 172.21.15.45:0/3548040851 conn(0x56389c635060 msgr2=0x56389c635440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3548040851 shutdown_connections 2024-09-10T10:08:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.245+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3548040851 wait complete. 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.246+0000 7f8fef1bf740 1 -- start start 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.246+0000 7f8fef1bf740 1 Processor -- start 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.246+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c640be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.246+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c641120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c641660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56389c646880 con 0x56389c639410 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56389c526290 con 0x56389c638af0 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56389c646a00 con 0x56389c643cb0 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fef1bf740 10 client.0 _ll_register_callbacks cb 0x56389c557140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c641120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c641120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55076/0 (socket says 172.21.15.45:55076) 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 -- 172.21.15.45:0/3919032369 learned_addr learned my addr 172.21.15.45:0/3919032369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fea9c9640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c641660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 -- 172.21.15.45:0/3919032369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 msgr2=0x56389c641660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c641660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 -- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 msgr2=0x56389c640be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c640be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c640be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 -- 172.21.15.45:0/3919032369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389c2b0210 con 0x56389c639410 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c640be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c641120 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0015ba0 tx=0x7f8fe00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4da67016a13f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.247+0000 7f8fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:08:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.248+0000 7f8fcdffb640 1 -- 172.21.15.45:0/3919032369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0025020 con 0x56389c639410 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.248+0000 7f8fcdffb640 1 -- 172.21.15.45:0/3919032369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f8fe0013a50 con 0x56389c639410 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.248+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fdc017070 con 0x56389c639410 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.248+0000 7f8fcdffb640 1 -- 172.21.15.45:0/3919032369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8fe001b040 con 0x56389c639410 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.248+0000 7f8fef1bf740 10 client.15471 fetch_fsmap learned FSMap version 21 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.248+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56389c37c620 con 0x56389c639410 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fcdffb640 1 -- 172.21.15.45:0/3919032369 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fdc017070 con 0x56389c639410 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fcdffb640 1 -- 172.21.15.45:0/3919032369 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8fe0032020 con 0x56389c639410 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fef1bf740 10 client.15471 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fef1bf740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fef1bf740 20 client.15471 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c653db0 0x56389c656200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fef1bf740 4 client.15471 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.249+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56389c5aaa20 con 0x56389c653db0 2024-09-10T10:08:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.251+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c653db0 0x56389c656200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.251+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c653db0 0x56389c656200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227128865c72ef5c server_cookie=af1fd7fceabda6f5 in_seq=0 out_seq=0 2024-09-10T10:08:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.251+0000 7f8fcdffb640 10 client.15471 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fcdffb640 1 -- 172.21.15.45:0/3919032369 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56389c5aaa20 con 0x56389c653db0 2024-09-10T10:08:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fcdffb640 10 client.15471 handle_command_reply: tid=0 2024-09-10T10:08:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 client.15471 shutdown 2024-09-10T10:08:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 msgr2=0x56389c641120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c641120 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0015ba0 tx=0x7f8fe00047d0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 shutdown_connections 2024-09-10T10:08:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c653db0 0x56389c656200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fcdffb640 0 client.15471 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c641660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c641120 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/3919032369 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c640be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.255+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 >> 172.21.15.45:0/3919032369 conn(0x56389c635060 msgr2=0x56389c63ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.256+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 shutdown_connections 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.256+0000 7f8fef1bf740 1 -- 172.21.15.45:0/3919032369 wait complete. 2024-09-10T10:08:43.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.256+0000 7f8fef1bf740 20 client.15471 trim_cache size 0 max 16384 2024-09-10T10:08:43.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.285+0000 7f8fef1bf740 1 Processor -- start 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.285+0000 7f8fef1bf740 1 -- start start 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.286+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c638af0 0x56389c641490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.286+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c639410 0x56389c642b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.286+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c65bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.286+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800c7c0 con 0x56389c643cb0 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.286+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800f060 con 0x56389c639410 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.286+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800ce50 con 0x56389c638af0 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.287+0000 7f8fe99c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c639410 0x56389c642b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.287+0000 7f8fe99c7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c639410 0x56389c642b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:35778/0 (socket says 172.21.15.45:35778) 2024-09-10T10:08:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.287+0000 7f8fe99c7640 1 -- 172.21.15.45:0/1297495418 learned_addr learned my addr 172.21.15.45:0/1297495418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.289+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c638af0 0x56389c641490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.293+0000 7f8fea9c9640 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c65bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.293+0000 7f8fea1c8640 1 -- 172.21.15.45:0/1297495418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c639410 msgr2=0x56389c642b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.293+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c639410 0x56389c642b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.293+0000 7f8fea1c8640 1 -- 172.21.15.45:0/1297495418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 msgr2=0x56389c65bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.293+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c65bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.293+0000 7f8fea1c8640 1 -- 172.21.15.45:0/1297495418 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389c635d50 con 0x56389c638af0 2024-09-10T10:08:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.297+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c638af0 0x56389c641490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40f6480 tx=0x7f8fd4004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=439f387f85f0df51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:43.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.299+0000 7f8fcffff640 1 -- 172.21.15.45:0/1297495418 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd404f040 con 0x56389c638af0 2024-09-10T10:08:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.299+0000 7f8fcffff640 1 -- 172.21.15.45:0/1297495418 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd40cdec0 con 0x56389c638af0 2024-09-10T10:08:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.300+0000 7f8fef1bf740 1 -- 172.21.15.45:0/1297495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c638af0 msgr2=0x56389c641490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:43.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.300+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c638af0 0x56389c641490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8fd40f6480 tx=0x7f8fd4004fb0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.303+0000 7f8fef1bf740 1 -- 172.21.15.45:0/1297495418 shutdown_connections 2024-09-10T10:08:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.303+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c643cb0 0x56389c65bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.303+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c639410 0x56389c642b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.303+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/1297495418 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c638af0 0x56389c641490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.303+0000 7f8fef1bf740 1 -- 172.21.15.45:0/1297495418 >> 172.21.15.45:0/1297495418 conn(0x56389c635060 msgr2=0x56389c642440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.303+0000 7f8fef1bf740 1 -- 172.21.15.45:0/1297495418 shutdown_connections 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.304+0000 7f8fef1bf740 1 -- 172.21.15.45:0/1297495418 wait complete. 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.304+0000 7f8fef1bf740 1 -- start start 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.304+0000 7f8fef1bf740 1 Processor -- start 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.304+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c64c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.304+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c63d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fef1bf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c63dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800d520 con 0x56389c639410 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800e990 con 0x56389c638af0 2024-09-10T10:08:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fef1bf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc800dbf0 con 0x56389c643cb0 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fef1bf740 10 client.0 _ll_register_callbacks cb 0x7f8fe00157f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fe99c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c63d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fe99c7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c63d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55082/0 (socket says 172.21.15.45:55082) 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fe99c7640 1 -- 172.21.15.45:0/2260893927 learned_addr learned my addr 172.21.15.45:0/2260893927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.305+0000 7f8fea9c9640 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c63dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.306+0000 7f8fe99c7640 1 -- 172.21.15.45:0/2260893927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 msgr2=0x56389c63dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.306+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c63dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.306+0000 7f8fe99c7640 1 -- 172.21.15.45:0/2260893927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 msgr2=0x56389c64c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.306+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c64c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.306+0000 7f8fe99c7640 1 -- 172.21.15.45:0/2260893927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389c63e160 con 0x56389c639410 2024-09-10T10:08:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.306+0000 7f8fe99c7640 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c63d660 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8fe002a5b0 tx=0x7f8fe002ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5decd86059315456 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.311+0000 7f8fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:08:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.311+0000 7f8fcdffb640 1 -- 172.21.15.45:0/2260893927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0037020 con 0x56389c639410 2024-09-10T10:08:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.311+0000 7f8fcdffb640 1 -- 172.21.15.45:0/2260893927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f8fe0021470 con 0x56389c639410 2024-09-10T10:08:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.311+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd40cdec0 con 0x56389c639410 2024-09-10T10:08:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.311+0000 7f8fcdffb640 1 -- 172.21.15.45:0/2260893927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8fe0042050 con 0x56389c639410 2024-09-10T10:08:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.311+0000 7f8fef1bf740 10 client.15477 fetch_fsmap learned FSMap version 21 2024-09-10T10:08:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.311+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56389c6b77d0 con 0x56389c639410 2024-09-10T10:08:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.313+0000 7f8fcdffb640 1 -- 172.21.15.45:0/2260893927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd40cdec0 con 0x56389c639410 2024-09-10T10:08:43.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.313+0000 7f8fcdffb640 1 -- 172.21.15.45:0/2260893927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f8fe0032040 con 0x56389c639410 2024-09-10T10:08:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.314+0000 7f8fef1bf740 10 client.15477 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:08:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.314+0000 7f8fef1bf740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.314+0000 7f8fef1bf740 20 client.15477 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.314+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c6b7f80 0x56389c6ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.314+0000 7f8fef1bf740 4 client.15477 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:08:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.314+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56389c5a4d70 con 0x56389c6b7f80 2024-09-10T10:08:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.317+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c6b7f80 0x56389c6ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.322+0000 7f8fea1c8640 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c6b7f80 0x56389c6ba3a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7398db17224dd877 server_cookie=2428f2dc681d6ffb in_seq=0 out_seq=0 2024-09-10T10:08:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:43.322+0000 7f8fcdffb640 10 client.15477 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:43.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:42 smithi162 ceph-mon[34963]: pgmap v169: 97 pgs: 97 active+clean; 50 GiB data, 269 GiB used, 803 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 751 MiB/s wr, 1.57k op/s 2024-09-10T10:08:43.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:42 smithi045 ceph-mon[31114]: pgmap v169: 97 pgs: 97 active+clean; 50 GiB data, 269 GiB used, 803 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 751 MiB/s wr, 1.57k op/s 2024-09-10T10:08:43.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:42 smithi047 ceph-mon[35244]: pgmap v169: 97 pgs: 97 active+clean; 50 GiB data, 269 GiB used, 803 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 751 MiB/s wr, 1.57k op/s 2024-09-10T10:08:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.781+0000 7f8fcdffb640 1 -- 172.21.15.45:0/2260893927 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56389c5a4d70 con 0x56389c6b7f80 2024-09-10T10:08:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.781+0000 7f8fcdffb640 10 client.15477 handle_command_reply: tid=0 2024-09-10T10:08:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.781+0000 7f8fef1bf740 1 client.15477 shutdown 2024-09-10T10:08:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.785+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 msgr2=0x56389c63d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.785+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c63d660 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8fe002a5b0 tx=0x7f8fe002ae50 comp rx=0 tx=0).stop 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.785+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 shutdown_connections 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.785+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56389c6b7f80 0x56389c6ba3a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.786+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56389c643cb0 0x56389c63dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.786+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56389c639410 0x56389c63d660 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.786+0000 7f8fef1bf740 1 --2- 172.21.15.45:0/2260893927 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56389c638af0 0x56389c64c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.786+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 >> 172.21.15.45:0/2260893927 conn(0x56389c635060 msgr2=0x56389c6341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.786+0000 7f8fcdffb640 0 client.15477 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:44.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.786+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 shutdown_connections 2024-09-10T10:08:44.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.786+0000 7f8fef1bf740 1 -- 172.21.15.45:0/2260893927 wait complete. 2024-09-10T10:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.789+0000 7f8fef1bf740 20 client.15477 trim_cache size 0 max 16384 2024-09-10T10:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:08:44.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8fd00a2e20 msgr2=0x7f8fd00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8fd00a2e20 0x7f8fd00a52e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fe406d9e0 tx=0x7f8fe0009110 comp rx=0 tx=0).stop 2024-09-10T10:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073790 msgr2=0x7f8fe41a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073790 0x7f8fe41a46e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4006010 tx=0x7f8fd4004e40 comp rx=0 tx=0).stop 2024-09-10T10:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 shutdown_connections 2024-09-10T10:08:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8fd00a2e20 0x7f8fd00a52e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fe41091f0 0x7f8fe419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fe4073c00 0x7f8fe41a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.793+0000 7f8fec42c640 1 --2- 172.21.15.45:0/2492553542 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8fe4073790 0x7f8fe41a46e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.794+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 >> 172.21.15.45:0/2492553542 conn(0x7f8fe406d210 msgr2=0x7f8fe410a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.794+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 shutdown_connections 2024-09-10T10:08:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:44.813+0000 7f8fec42c640 1 -- 172.21.15.45:0/2492553542 wait complete. 2024-09-10T10:08:45.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:45 smithi162 ceph-mon[34963]: pgmap v170: 97 pgs: 97 active+clean; 50 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 950 MiB/s wr, 1.74k op/s 2024-09-10T10:08:45.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:45 smithi045 ceph-mon[31114]: pgmap v170: 97 pgs: 97 active+clean; 50 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 950 MiB/s wr, 1.74k op/s 2024-09-10T10:08:45.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:45 smithi047 ceph-mon[35244]: pgmap v170: 97 pgs: 97 active+clean; 50 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 950 MiB/s wr, 1.74k op/s 2024-09-10T10:08:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:47 smithi162 ceph-mon[34963]: pgmap v171: 97 pgs: 97 active+clean; 50 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 775 MiB/s wr, 1.44k op/s 2024-09-10T10:08:47.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:47 smithi045 ceph-mon[31114]: pgmap v171: 97 pgs: 97 active+clean; 50 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 775 MiB/s wr, 1.44k op/s 2024-09-10T10:08:47.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:47 smithi047 ceph-mon[35244]: pgmap v171: 97 pgs: 97 active+clean; 50 GiB data, 274 GiB used, 799 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 775 MiB/s wr, 1.44k op/s 2024-09-10T10:08:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:49 smithi162 ceph-mon[34963]: pgmap v172: 97 pgs: 97 active+clean; 50 GiB data, 275 GiB used, 798 GiB / 1.0 TiB avail; 4.5 MiB/s rd, 861 MiB/s wr, 1.64k op/s 2024-09-10T10:08:49.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:49 smithi045 ceph-mon[31114]: pgmap v172: 97 pgs: 97 active+clean; 50 GiB data, 275 GiB used, 798 GiB / 1.0 TiB avail; 4.5 MiB/s rd, 861 MiB/s wr, 1.64k op/s 2024-09-10T10:08:49.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:49 smithi047 ceph-mon[35244]: pgmap v172: 97 pgs: 97 active+clean; 50 GiB data, 275 GiB used, 798 GiB / 1.0 TiB avail; 4.5 MiB/s rd, 861 MiB/s wr, 1.64k op/s 2024-09-10T10:08:51.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:51 smithi162 ceph-mon[34963]: pgmap v173: 97 pgs: 97 active+clean; 50 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 869 MiB/s wr, 1.55k op/s 2024-09-10T10:08:51.657 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:51 smithi045 ceph-mon[31114]: pgmap v173: 97 pgs: 97 active+clean; 50 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 869 MiB/s wr, 1.55k op/s 2024-09-10T10:08:51.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:51 smithi047 ceph-mon[35244]: pgmap v173: 97 pgs: 97 active+clean; 50 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 869 MiB/s wr, 1.55k op/s 2024-09-10T10:08:53.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:53 smithi162 ceph-mon[34963]: pgmap v174: 97 pgs: 97 active+clean; 50 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 746 MiB/s wr, 1.16k op/s 2024-09-10T10:08:53.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:53 smithi045 ceph-mon[31114]: pgmap v174: 97 pgs: 97 active+clean; 50 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 746 MiB/s wr, 1.16k op/s 2024-09-10T10:08:53.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:53 smithi047 ceph-mon[35244]: pgmap v174: 97 pgs: 97 active+clean; 50 GiB data, 278 GiB used, 795 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 746 MiB/s wr, 1.16k op/s 2024-09-10T10:08:55.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:55 smithi162 ceph-mon[34963]: pgmap v175: 97 pgs: 97 active+clean; 50 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 928 MiB/s wr, 1.34k op/s 2024-09-10T10:08:55.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:55 smithi045 ceph-mon[31114]: pgmap v175: 97 pgs: 97 active+clean; 50 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 928 MiB/s wr, 1.34k op/s 2024-09-10T10:08:55.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:55 smithi047 ceph-mon[35244]: pgmap v175: 97 pgs: 97 active+clean; 50 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 928 MiB/s wr, 1.34k op/s 2024-09-10T10:08:57.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:57 smithi162 ceph-mon[34963]: pgmap v176: 97 pgs: 97 active+clean; 50 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 763 MiB/s wr, 1.22k op/s 2024-09-10T10:08:57.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:57 smithi045 ceph-mon[31114]: pgmap v176: 97 pgs: 97 active+clean; 50 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 763 MiB/s wr, 1.22k op/s 2024-09-10T10:08:57.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:57 smithi047 ceph-mon[35244]: pgmap v176: 97 pgs: 97 active+clean; 50 GiB data, 282 GiB used, 791 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 763 MiB/s wr, 1.22k op/s 2024-09-10T10:08:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.099+0000 7f264b511640 1 -- 172.21.15.45:0/2875667714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644112af0 msgr2=0x7f2644114ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.099+0000 7f264b511640 1 --2- 172.21.15.45:0/2875667714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644112af0 0x7f2644114ee0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f263c01acb0 tx=0x7f263c03b9b0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.099+0000 7f264b511640 1 -- 172.21.15.45:0/2875667714 shutdown_connections 2024-09-10T10:08:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.099+0000 7f264b511640 1 --2- 172.21.15.45:0/2875667714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644112af0 0x7f2644114ee0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.099+0000 7f264b511640 1 --2- 172.21.15.45:0/2875667714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2644074eb0 0x7f26441125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.099+0000 7f264b511640 1 --2- 172.21.15.45:0/2875667714 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2644074590 0x7f2644074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.099+0000 7f264b511640 1 -- 172.21.15.45:0/2875667714 >> 172.21.15.45:0/2875667714 conn(0x7f26440fd7e0 msgr2=0x7f26440ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.100+0000 7f264b511640 1 -- 172.21.15.45:0/2875667714 shutdown_connections 2024-09-10T10:08:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.100+0000 7f264b511640 1 -- 172.21.15.45:0/2875667714 wait complete. 2024-09-10T10:08:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.101+0000 7f264b511640 1 Processor -- start 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.101+0000 7f264b511640 1 -- start start 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.101+0000 7f264b511640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644074590 0x7f26441122c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.101+0000 7f264b511640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2644074eb0 0x7f264410e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.102+0000 7f264b511640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2644112af0 0x7f264410e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.102+0000 7f264b511640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2644108840 con 0x7f2644074590 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.102+0000 7f264b511640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26441086c0 con 0x7f2644112af0 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.102+0000 7f26492ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644074590 0x7f26441122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.102+0000 7f26492ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644074590 0x7f26441122c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60224/0 (socket says 172.21.15.45:60224) 2024-09-10T10:08:58.105 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.102+0000 7f26492ad640 1 -- 172.21.15.45:0/491225791 learned_addr learned my addr 172.21.15.45:0/491225791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.102+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f26441089c0 con 0x7f2644074eb0 2024-09-10T10:08:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.103+0000 7f2648aac640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2644074eb0 0x7f264410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.103+0000 7f26492ad640 1 -- 172.21.15.45:0/491225791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2644074eb0 msgr2=0x7f264410e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.103+0000 7f26492ad640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2644074eb0 0x7f264410e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.103+0000 7f26492ad640 1 -- 172.21.15.45:0/491225791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2644112af0 msgr2=0x7f264410e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.103+0000 7f26492ad640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2644112af0 0x7f264410e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.103+0000 7f26492ad640 1 -- 172.21.15.45:0/491225791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264410f010 con 0x7f2644074590 2024-09-10T10:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.104+0000 7f26492ad640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644074590 0x7f26441122c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f263400b450 tx=0x7f263400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.104+0000 7f263a7fc640 1 -- 172.21.15.45:0/491225791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634015020 con 0x7f2644074590 2024-09-10T10:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.105+0000 7f263a7fc640 1 -- 172.21.15.45:0/491225791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2634011d70 con 0x7f2644074590 2024-09-10T10:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.105+0000 7f263a7fc640 1 -- 172.21.15.45:0/491225791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263400f340 con 0x7f2644074590 2024-09-10T10:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.106+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26441a28c0 con 0x7f2644074590 2024-09-10T10:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.106+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441a2d30 con 0x7f2644074590 2024-09-10T10:08:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.113+0000 7f263a7fc640 1 -- 172.21.15.45:0/491225791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2634018020 con 0x7f2644074590 2024-09-10T10:08:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.115+0000 7f264e2ac740 1 Processor -- start 2024-09-10T10:08:58.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.115+0000 7f264e2ac740 1 -- start start 2024-09-10T10:08:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.116+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.116+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a32d0 0x5588ca3be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.116+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3becc0 0x5588ca3c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.116+0000 7f264e2ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588ca3072f0 con 0x5588ca3becc0 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.116+0000 7f264e2ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5588ca2af270 con 0x5588ca3a29b0 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.116+0000 7f264e2ac740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5588ca3032b0 con 0x5588ca3a32d0 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.118+0000 7f2648aac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3becc0 0x5588ca3c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.118+0000 7f2648aac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3becc0 0x5588ca3c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60240/0 (socket says 172.21.15.45:60240) 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.118+0000 7f2648aac640 1 -- 172.21.15.45:0/1165386002 learned_addr learned my addr 172.21.15.45:0/1165386002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.119+0000 7f2648aac640 1 -- 172.21.15.45:0/1165386002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a32d0 msgr2=0x5588ca3be780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.119+0000 7f2648aac640 1 --2- 172.21.15.45:0/1165386002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a32d0 0x5588ca3be780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.119+0000 7f2648aac640 1 -- 172.21.15.45:0/1165386002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 msgr2=0x5588ca3a2d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.119+0000 7f2648aac640 1 --2- 172.21.15.45:0/1165386002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3a2d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.119+0000 7f2648aac640 1 -- 172.21.15.45:0/1165386002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ca2ae340 con 0x5588ca3becc0 2024-09-10T10:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.119+0000 7f2648aac640 1 --2- 172.21.15.45:0/1165386002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3becc0 0x5588ca3c10b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f264002d960 tx=0x7f264002de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33c589890ca5a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.120+0000 7f262f7fe640 1 -- 172.21.15.45:0/1165386002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640038020 con 0x5588ca3becc0 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.120+0000 7f262f7fe640 1 -- 172.21.15.45:0/1165386002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26400343f0 con 0x5588ca3becc0 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.120+0000 7f262f7fe640 1 -- 172.21.15.45:0/1165386002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640030040 con 0x5588ca3becc0 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.121+0000 7f264e2ac740 1 -- 172.21.15.45:0/1165386002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3becc0 msgr2=0x5588ca3c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.121+0000 7f264e2ac740 1 --2- 172.21.15.45:0/1165386002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3becc0 0x5588ca3c10b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f264002d960 tx=0x7f264002de20 comp rx=0 tx=0).stop 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.122+0000 7f264e2ac740 1 -- 172.21.15.45:0/1165386002 shutdown_connections 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.122+0000 7f264e2ac740 1 --2- 172.21.15.45:0/1165386002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3becc0 0x5588ca3c10b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.122+0000 7f264e2ac740 1 --2- 172.21.15.45:0/1165386002 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a32d0 0x5588ca3be780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.122+0000 7f264e2ac740 1 --2- 172.21.15.45:0/1165386002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.122+0000 7f264e2ac740 1 -- 172.21.15.45:0/1165386002 >> 172.21.15.45:0/1165386002 conn(0x5588ca39f060 msgr2=0x5588ca39f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.124+0000 7f263a7fc640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f26300a2f80 0x7f26300a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.125+0000 7f263a7fc640 1 -- 172.21.15.45:0/491225791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f26340cec40 con 0x7f2644074590 2024-09-10T10:08:58.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.127+0000 7f2648aac640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f26300a2f80 0x7f26300a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.135+0000 7f2648aac640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f26300a2f80 0x7f26300a5440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2640034d40 tx=0x7f2640000bc0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.143+0000 7f264e2ac740 1 -- 172.21.15.45:0/1165386002 shutdown_connections 2024-09-10T10:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.146+0000 7f264e2ac740 1 -- 172.21.15.45:0/1165386002 wait complete. 2024-09-10T10:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.150+0000 7f264e2ac740 1 -- start start 2024-09-10T10:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.150+0000 7f264e2ac740 1 Processor -- start 2024-09-10T10:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.151+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.151+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.151+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3becc0 0x5588ca3ab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.151+0000 7f264e2ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588ca3a5950 con 0x5588ca3a32d0 2024-09-10T10:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.151+0000 7f264e2ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5588ca290290 con 0x5588ca3becc0 2024-09-10T10:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.151+0000 7f264e2ac740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5588ca3a5ad0 con 0x5588ca3a29b0 2024-09-10T10:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.151+0000 7f264e2ac740 10 client.0 _ll_register_callbacks cb 0x5588ca2c1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.152+0000 7f2649aae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3becc0 0x5588ca3ab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.152+0000 7f2649aae640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3becc0 0x5588ca3ab3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:60004/0 (socket says 172.21.15.45:60004) 2024-09-10T10:08:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.152+0000 7f2649aae640 1 -- 172.21.15.45:0/2172353653 learned_addr learned my addr 172.21.15.45:0/2172353653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.153+0000 7f2649aae640 1 -- 172.21.15.45:0/2172353653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a29b0 msgr2=0x5588ca3b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.153+0000 7f2649aae640 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.153+0000 7f2649aae640 1 -- 172.21.15.45:0/2172353653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 msgr2=0x5588ca3b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.153+0000 7f2649aae640 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.153+0000 7f2649aae640 1 -- 172.21.15.45:0/2172353653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ca01a210 con 0x5588ca3becc0 2024-09-10T10:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.154+0000 7f2649aae640 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3becc0 0x5588ca3ab3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f263c0476d0 tx=0x7f263c047700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6f9be1550ae649 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.160+0000 7f262cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:08:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.160+0000 7f262cff9640 1 -- 172.21.15.45:0/2172353653 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c056060 con 0x5588ca3becc0 2024-09-10T10:08:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.160+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26400343f0 con 0x5588ca3becc0 2024-09-10T10:08:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.160+0000 7f262cff9640 1 -- 172.21.15.45:0/2172353653 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f263c003c40 con 0x5588ca3becc0 2024-09-10T10:08:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.161+0000 7f262cff9640 1 -- 172.21.15.45:0/2172353653 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f263c044040 con 0x5588ca3becc0 2024-09-10T10:08:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.161+0000 7f264e2ac740 10 client.25145 fetch_fsmap learned FSMap version 21 2024-09-10T10:08:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.161+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5588ca0e6620 con 0x5588ca3becc0 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f262cff9640 1 -- 172.21.15.45:0/2172353653 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26400343f0 con 0x5588ca3becc0 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f262cff9640 1 -- 172.21.15.45:0/2172353653 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f263c004420 con 0x5588ca3becc0 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f264e2ac740 10 client.25145 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f264e2ac740 10 client.25145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f264e2ac740 20 client.25145 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f264e2ac740 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca3bb810 0x5588ca3bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f264e2ac740 4 client.25145 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:08:58.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.162+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588ca314a20 con 0x5588ca3bb810 2024-09-10T10:08:58.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.164+0000 7f26492ad640 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca3bb810 0x5588ca3bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.182+0000 7f262cff9640 10 client.25145 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.182+0000 7f26492ad640 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca3bb810 0x5588ca3bdcd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673739ae987caa0f server_cookie=c12ff2d4612203ee in_seq=0 out_seq=0 2024-09-10T10:08:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.203+0000 7f262cff9640 1 -- 172.21.15.45:0/2172353653 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588ca314a20 con 0x5588ca3bb810 2024-09-10T10:08:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.203+0000 7f262cff9640 10 client.25145 handle_command_reply: tid=0 2024-09-10T10:08:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.203+0000 7f264e2ac740 1 client.25145 shutdown 2024-09-10T10:08:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3becc0 msgr2=0x5588ca3ab3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3becc0 0x5588ca3ab3c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f263c0476d0 tx=0x7f263c047700 comp rx=0 tx=0).stop 2024-09-10T10:08:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 shutdown_connections 2024-09-10T10:08:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca3bb810 0x5588ca3bdcd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3becc0 0x5588ca3ab3c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 --2- 172.21.15.45:0/2172353653 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 >> 172.21.15.45:0/2172353653 conn(0x5588ca39f060 msgr2=0x5588ca3a4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.205+0000 7f262cff9640 0 client.25145 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.207+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 shutdown_connections 2024-09-10T10:08:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.207+0000 7f264e2ac740 1 -- 172.21.15.45:0/2172353653 wait complete. 2024-09-10T10:08:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.207+0000 7f264e2ac740 20 client.25145 trim_cache size 0 max 16384 2024-09-10T10:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.269+0000 7f264e2ac740 1 Processor -- start 2024-09-10T10:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.273+0000 7f264e2ac740 1 -- start start 2024-09-10T10:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.274+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.274+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3aac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.274+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.275+0000 7f264e2ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f26440749f0 con 0x5588ca3a32d0 2024-09-10T10:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.275+0000 7f264e2ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26441a37e0 con 0x5588ca3a29b0 2024-09-10T10:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.275+0000 7f264e2ac740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2644072a00 con 0x5588ca3becc0 2024-09-10T10:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.275+0000 7f2648aac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.276+0000 7f2648aac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60258/0 (socket says 172.21.15.45:60258) 2024-09-10T10:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.276+0000 7f2648aac640 1 -- 172.21.15.45:0/683217416 learned_addr learned my addr 172.21.15.45:0/683217416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:58.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.276+0000 7f26492ad640 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.279+0000 7f2649aae640 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.281+0000 7f26492ad640 1 -- 172.21.15.45:0/683217416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 msgr2=0x5588ca3ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.281+0000 7f26492ad640 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.281+0000 7f26492ad640 1 -- 172.21.15.45:0/683217416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 msgr2=0x5588ca3aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.281+0000 7f26492ad640 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3aac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.281+0000 7f26492ad640 1 -- 172.21.15.45:0/683217416 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588ca3a3100 con 0x5588ca3a29b0 2024-09-10T10:08:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.290+0000 7f26492ad640 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26340cc470 tx=0x7f26340ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70567e12bbc3b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.295+0000 7f262effd640 1 -- 172.21.15.45:0/683217416 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263409f5f0 con 0x5588ca3a29b0 2024-09-10T10:08:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.295+0000 7f262effd640 1 -- 172.21.15.45:0/683217416 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26340a3eb0 con 0x5588ca3a29b0 2024-09-10T10:08:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.295+0000 7f262effd640 1 -- 172.21.15.45:0/683217416 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340978b0 con 0x5588ca3a29b0 2024-09-10T10:08:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.297+0000 7f264e2ac740 1 -- 172.21.15.45:0/683217416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 msgr2=0x5588ca3b0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:08:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.297+0000 7f264e2ac740 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26340cc470 tx=0x7f26340ce040 comp rx=0 tx=0).stop 2024-09-10T10:08:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.298+0000 7f264e2ac740 1 -- 172.21.15.45:0/683217416 shutdown_connections 2024-09-10T10:08:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.298+0000 7f264e2ac740 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.298+0000 7f264e2ac740 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.298+0000 7f264e2ac740 1 --2- 172.21.15.45:0/683217416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b0570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.298+0000 7f264e2ac740 1 -- 172.21.15.45:0/683217416 >> 172.21.15.45:0/683217416 conn(0x5588ca39f060 msgr2=0x5588ca3afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:08:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.298+0000 7f264e2ac740 1 -- 172.21.15.45:0/683217416 shutdown_connections 2024-09-10T10:08:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.298+0000 7f264e2ac740 1 -- 172.21.15.45:0/683217416 wait complete. 2024-09-10T10:08:58.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.299+0000 7f264e2ac740 1 -- start start 2024-09-10T10:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.299+0000 7f264e2ac740 1 Processor -- start 2024-09-10T10:08:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.302+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.302+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.303+0000 7f264e2ac740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.303+0000 7f264e2ac740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f26440730d0 con 0x5588ca3a32d0 2024-09-10T10:08:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.303+0000 7f264e2ac740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26441a3110 con 0x5588ca3a29b0 2024-09-10T10:08:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.303+0000 7f264e2ac740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f26440737a0 con 0x5588ca3becc0 2024-09-10T10:08:58.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.303+0000 7f264e2ac740 10 client.0 _ll_register_callbacks cb 0x7f263c01ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:08:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.304+0000 7f2648aac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.304+0000 7f2648aac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60260/0 (socket says 172.21.15.45:60260) 2024-09-10T10:08:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.304+0000 7f2648aac640 1 -- 172.21.15.45:0/455923918 learned_addr learned my addr 172.21.15.45:0/455923918 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:08:58.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.305+0000 7f26492ad640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.306+0000 7f2649aae640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.306+0000 7f26492ad640 1 -- 172.21.15.45:0/455923918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 msgr2=0x5588ca3b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.306+0000 7f26492ad640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.306+0000 7f26492ad640 1 -- 172.21.15.45:0/455923918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 msgr2=0x5588ca3b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.306+0000 7f26492ad640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.306+0000 7f26492ad640 1 -- 172.21.15.45:0/455923918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588ca427740 con 0x5588ca3a29b0 2024-09-10T10:08:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.306+0000 7f2649aae640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:08:58.315 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.307+0000 7f26492ad640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b2130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f263409b120 tx=0x7f2634010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40344ba8d1d660a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:08:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.317+0000 7f262cff9640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:08:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.317+0000 7f262cff9640 1 -- 172.21.15.45:0/455923918 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340ae070 con 0x5588ca3a29b0 2024-09-10T10:08:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.317+0000 7f262cff9640 1 -- 172.21.15.45:0/455923918 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f2634099540 con 0x5588ca3a29b0 2024-09-10T10:08:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.318+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26340a3eb0 con 0x5588ca3a29b0 2024-09-10T10:08:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.318+0000 7f262cff9640 1 -- 172.21.15.45:0/455923918 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f26340b4120 con 0x5588ca3a29b0 2024-09-10T10:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.324+0000 7f264e2ac740 10 client.25151 fetch_fsmap learned FSMap version 21 2024-09-10T10:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.324+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5588ca4289f0 con 0x5588ca3a29b0 2024-09-10T10:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.324+0000 7f262cff9640 1 -- 172.21.15.45:0/455923918 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26340a6030 con 0x5588ca3a29b0 2024-09-10T10:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.325+0000 7f262cff9640 1 -- 172.21.15.45:0/455923918 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2634022060 con 0x5588ca3a29b0 2024-09-10T10:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.325+0000 7f264e2ac740 10 client.25151 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.325+0000 7f264e2ac740 10 client.25151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:08:58.326 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.325+0000 7f264e2ac740 20 client.25151 populate_metadata read hostname 'smithi045' 2024-09-10T10:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.325+0000 7f264e2ac740 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca428f20 0x5588ca42b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.325+0000 7f264e2ac740 4 client.25151 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:08:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.325+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588ca30ed70 con 0x5588ca428f20 2024-09-10T10:08:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.326+0000 7f2648aac640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca428f20 0x5588ca42b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:08:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.335+0000 7f262cff9640 10 client.25151 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:08:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:08:58.335+0000 7f2648aac640 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca428f20 0x5588ca42b310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0037076438e398e server_cookie=83e6d37dbef4ad9e in_seq=0 out_seq=0 2024-09-10T10:08:59.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:08:59 smithi162 ceph-mon[34963]: pgmap v177: 97 pgs: 97 active+clean; 50 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 854 MiB/s wr, 1.38k op/s 2024-09-10T10:08:59.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:08:59 smithi045 ceph-mon[31114]: pgmap v177: 97 pgs: 97 active+clean; 50 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 854 MiB/s wr, 1.38k op/s 2024-09-10T10:08:59.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:08:59 smithi047 ceph-mon[35244]: pgmap v177: 97 pgs: 97 active+clean; 50 GiB data, 283 GiB used, 790 GiB / 1.0 TiB avail; 3.4 MiB/s rd, 854 MiB/s wr, 1.38k op/s 2024-09-10T10:09:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.772+0000 7f262cff9640 1 -- 172.21.15.45:0/455923918 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588ca30ed70 con 0x5588ca428f20 2024-09-10T10:09:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.772+0000 7f262cff9640 10 client.25151 handle_command_reply: tid=0 2024-09-10T10:09:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.772+0000 7f264e2ac740 1 client.25151 shutdown 2024-09-10T10:09:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.773+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 msgr2=0x5588ca3b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.773+0000 7f264e2ac740 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b2130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f263409b120 tx=0x7f2634010990 comp rx=0 tx=0).stop 2024-09-10T10:09:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.778+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 shutdown_connections 2024-09-10T10:09:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.778+0000 7f264e2ac740 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5588ca428f20 0x5588ca42b310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.778+0000 7f264e2ac740 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5588ca3becc0 0x5588ca3b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.778+0000 7f264e2ac740 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588ca3a32d0 0x5588ca3b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.778+0000 7f264e2ac740 1 --2- 172.21.15.45:0/455923918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5588ca3a29b0 0x5588ca3b2130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.778+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 >> 172.21.15.45:0/455923918 conn(0x5588ca39f060 msgr2=0x5588ca3aabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.778+0000 7f262cff9640 0 client.25151 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.781+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 shutdown_connections 2024-09-10T10:09:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.781+0000 7f264e2ac740 1 -- 172.21.15.45:0/455923918 wait complete. 2024-09-10T10:09:01.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.782+0000 7f264e2ac740 20 client.25151 trim_cache size 0 max 16384 2024-09-10T10:09:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:09:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:09:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:09:01.784 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:09:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f26300a2f80 msgr2=0x7f26300a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f26300a2f80 0x7f26300a5440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2640034d40 tx=0x7f2640000bc0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644074590 msgr2=0x7f26441122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644074590 0x7f26441122c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f263400b450 tx=0x7f263400b910 comp rx=0 tx=0).stop 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 shutdown_connections 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f26300a2f80 0x7f26300a5440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2644112af0 0x7f264410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2644074eb0 0x7f264410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 --2- 172.21.15.45:0/491225791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2644074590 0x7f26441122c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.785+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 >> 172.21.15.45:0/491225791 conn(0x7f26440fd7e0 msgr2=0x7f26440fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.795+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 shutdown_connections 2024-09-10T10:09:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:01.797+0000 7f264b511640 1 -- 172.21.15.45:0/491225791 wait complete. 2024-09-10T10:09:01.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:01 smithi045 ceph-mon[31114]: pgmap v178: 97 pgs: 97 active+clean; 50 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 867 MiB/s wr, 1.34k op/s 2024-09-10T10:09:01.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:01 smithi047 ceph-mon[35244]: pgmap v178: 97 pgs: 97 active+clean; 50 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 867 MiB/s wr, 1.34k op/s 2024-09-10T10:09:02.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:01 smithi162 ceph-mon[34963]: pgmap v178: 97 pgs: 97 active+clean; 50 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 867 MiB/s wr, 1.34k op/s 2024-09-10T10:09:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:03 smithi162 ceph-mon[34963]: pgmap v179: 97 pgs: 97 active+clean; 50 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 742 MiB/s wr, 1.08k op/s 2024-09-10T10:09:04.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:03 smithi047 ceph-mon[35244]: pgmap v179: 97 pgs: 97 active+clean; 50 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 742 MiB/s wr, 1.08k op/s 2024-09-10T10:09:04.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:03 smithi045 ceph-mon[31114]: pgmap v179: 97 pgs: 97 active+clean; 50 GiB data, 286 GiB used, 787 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 742 MiB/s wr, 1.08k op/s 2024-09-10T10:09:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:05 smithi162 ceph-mon[34963]: pgmap v180: 97 pgs: 97 active+clean; 50 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 4.6 MiB/s rd, 915 MiB/s wr, 1.37k op/s 2024-09-10T10:09:06.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:05 smithi045 ceph-mon[31114]: pgmap v180: 97 pgs: 97 active+clean; 50 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 4.6 MiB/s rd, 915 MiB/s wr, 1.37k op/s 2024-09-10T10:09:06.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:05 smithi047 ceph-mon[35244]: pgmap v180: 97 pgs: 97 active+clean; 50 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 4.6 MiB/s rd, 915 MiB/s wr, 1.37k op/s 2024-09-10T10:09:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:07 smithi162 ceph-mon[34963]: pgmap v181: 97 pgs: 97 active+clean; 50 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 756 MiB/s wr, 1.18k op/s 2024-09-10T10:09:08.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:07 smithi045 ceph-mon[31114]: pgmap v181: 97 pgs: 97 active+clean; 50 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 756 MiB/s wr, 1.18k op/s 2024-09-10T10:09:08.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:07 smithi047 ceph-mon[35244]: pgmap v181: 97 pgs: 97 active+clean; 50 GiB data, 290 GiB used, 783 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 756 MiB/s wr, 1.18k op/s 2024-09-10T10:09:10.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:09 smithi162 ceph-mon[34963]: pgmap v182: 97 pgs: 97 active+clean; 51 GiB data, 293 GiB used, 780 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 849 MiB/s wr, 1.87k op/s 2024-09-10T10:09:10.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:09 smithi045 ceph-mon[31114]: pgmap v182: 97 pgs: 97 active+clean; 51 GiB data, 293 GiB used, 780 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 849 MiB/s wr, 1.87k op/s 2024-09-10T10:09:10.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:09 smithi047 ceph-mon[35244]: pgmap v182: 97 pgs: 97 active+clean; 51 GiB data, 293 GiB used, 780 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 849 MiB/s wr, 1.87k op/s 2024-09-10T10:09:12.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:11 smithi162 ceph-mon[34963]: pgmap v183: 97 pgs: 97 active+clean; 52 GiB data, 297 GiB used, 776 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 859 MiB/s wr, 2.77k op/s 2024-09-10T10:09:12.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:11 smithi045 ceph-mon[31114]: pgmap v183: 97 pgs: 97 active+clean; 52 GiB data, 297 GiB used, 776 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 859 MiB/s wr, 2.77k op/s 2024-09-10T10:09:12.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:11 smithi047 ceph-mon[35244]: pgmap v183: 97 pgs: 97 active+clean; 52 GiB data, 297 GiB used, 776 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 859 MiB/s wr, 2.77k op/s 2024-09-10T10:09:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:13 smithi162 ceph-mon[34963]: pgmap v184: 97 pgs: 97 active+clean; 52 GiB data, 297 GiB used, 775 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 718 MiB/s wr, 2.81k op/s 2024-09-10T10:09:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:13 smithi045 ceph-mon[31114]: pgmap v184: 97 pgs: 97 active+clean; 52 GiB data, 297 GiB used, 775 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 718 MiB/s wr, 2.81k op/s 2024-09-10T10:09:14.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:13 smithi047 ceph-mon[35244]: pgmap v184: 97 pgs: 97 active+clean; 52 GiB data, 297 GiB used, 775 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 718 MiB/s wr, 2.81k op/s 2024-09-10T10:09:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 -- 172.21.15.45:0/3974432872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b40fff60 msgr2=0x7f96b41003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/3974432872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b40fff60 0x7f96b41003c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f969c009900 tx=0x7f969c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 -- 172.21.15.45:0/3974432872 shutdown_connections 2024-09-10T10:09:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/3974432872 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b41009e0 0x7f96b41046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/3974432872 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b40fff60 0x7f96b41003c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/3974432872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96b4106be0 0x7f96b4106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 -- 172.21.15.45:0/3974432872 >> 172.21.15.45:0/3974432872 conn(0x7f96b40770f0 msgr2=0x7f96b40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 -- 172.21.15.45:0/3974432872 shutdown_connections 2024-09-10T10:09:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.079+0000 7f96b9e4c640 1 -- 172.21.15.45:0/3974432872 wait complete. 2024-09-10T10:09:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.080+0000 7f96b9e4c640 1 Processor -- start 2024-09-10T10:09:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.080+0000 7f96b9e4c640 1 -- start start 2024-09-10T10:09:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96b9e4c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b40fff60 0x7f96b41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96b9e4c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b41009e0 0x7f96b41a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96b9e4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96b4106be0 0x7f96b419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96b9e4c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41185f0 con 0x7f96b4106be0 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96b9e4c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4118470 con 0x7f96b40fff60 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96b9e4c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4118770 con 0x7f96b41009e0 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96abfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b41009e0 0x7f96b41a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96abfff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b41009e0 0x7f96b41a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:55818/0 (socket says 172.21.15.45:55818) 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.081+0000 7f96abfff640 1 -- 172.21.15.45:0/1926833687 learned_addr learned my addr 172.21.15.45:0/1926833687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96b37fe640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b40fff60 0x7f96b41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96b3fff640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96b4106be0 0x7f96b419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96abfff640 1 -- 172.21.15.45:0/1926833687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b40fff60 msgr2=0x7f96b41a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96abfff640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b40fff60 0x7f96b41a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96abfff640 1 -- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96b4106be0 msgr2=0x7f96b419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96abfff640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96b4106be0 0x7f96b419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96abfff640 1 -- 172.21.15.45:0/1926833687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b419eff0 con 0x7f96b41009e0 2024-09-10T10:09:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.082+0000 7f96b3fff640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96b4106be0 0x7f96b419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:09:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.083+0000 7f96abfff640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b41009e0 0x7f96b41a4bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f969c004770 tx=0x7f969c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.089+0000 7f96b17fa640 1 -- 172.21.15.45:0/1926833687 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c03f070 con 0x7f96b41009e0 2024-09-10T10:09:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.090+0000 7f96b17fa640 1 -- 172.21.15.45:0/1926833687 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f969c031ca0 con 0x7f96b41009e0 2024-09-10T10:09:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.090+0000 7f96b17fa640 1 -- 172.21.15.45:0/1926833687 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c035030 con 0x7f96b41009e0 2024-09-10T10:09:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.091+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b419f280 con 0x7f96b41009e0 2024-09-10T10:09:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.091+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b419f660 con 0x7f96b41009e0 2024-09-10T10:09:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.095+0000 7f96bcbe6740 1 Processor -- start 2024-09-10T10:09:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.096+0000 7f96b17fa640 1 -- 172.21.15.45:0/1926833687 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f969c042070 con 0x7f96b41009e0 2024-09-10T10:09:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.096+0000 7f96bcbe6740 1 -- start start 2024-09-10T10:09:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b71bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71c410 0x557c6b726770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b726cb0 0x557c6b7290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557c6b6802f0 con 0x557c6b726cb0 2024-09-10T10:09:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557c6b628270 con 0x557c6b71baf0 2024-09-10T10:09:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557c6b67c2b0 con 0x557c6b71c410 2024-09-10T10:09:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b71bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71c410 0x557c6b726770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b71bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:33872/0 (socket says 172.21.15.45:33872) 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96b3fff640 1 -- 172.21.15.45:0/3268874330 learned_addr learned my addr 172.21.15.45:0/3268874330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.097+0000 7f96abfff640 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b726cb0 0x557c6b7290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.098+0000 7f96b3fff640 1 -- 172.21.15.45:0/3268874330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71c410 msgr2=0x557c6b726770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.098+0000 7f96b3fff640 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71c410 0x557c6b726770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.098+0000 7f96b3fff640 1 -- 172.21.15.45:0/3268874330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b726cb0 msgr2=0x557c6b7290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.098+0000 7f96b3fff640 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b726cb0 0x557c6b7290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.100 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.098+0000 7f96b3fff640 1 -- 172.21.15.45:0/3268874330 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6b627340 con 0x557c6b71baf0 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.098+0000 7f96b3fff640 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b71bed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96ac010750 tx=0x7f96ac010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985d1ba4301192e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.098+0000 7f96a9ffb640 1 -- 172.21.15.45:0/3268874330 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00ead0 con 0x557c6b71baf0 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.099+0000 7f96b17fa640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f96a00a2f00 0x7f96a00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.099+0000 7f96a9ffb640 1 -- 172.21.15.45:0/3268874330 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ac010e60 con 0x557c6b71baf0 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.099+0000 7f96b17fa640 1 -- 172.21.15.45:0/1926833687 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f969c0efbf0 con 0x7f96b41009e0 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.099+0000 7f96b37fe640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f96a00a2f00 0x7f96a00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.100+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3268874330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 msgr2=0x557c6b71bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.100+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b71bed0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f96ac010750 tx=0x7f96ac010c10 comp rx=0 tx=0).stop 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.100+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3268874330 shutdown_connections 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.100+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b726cb0 0x557c6b7290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.100+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71c410 0x557c6b726770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.100+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3268874330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b71bed0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.102 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.100+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3268874330 >> 172.21.15.45:0/3268874330 conn(0x557c6b718060 msgr2=0x557c6b718440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.101+0000 7f96b37fe640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f96a00a2f00 0x7f96a00a53c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x557c6b718ec0 tx=0x7f96a4005f90 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.101+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3268874330 shutdown_connections 2024-09-10T10:09:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.101+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3268874330 wait complete. 2024-09-10T10:09:15.103 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.101+0000 7f96bcbe6740 1 -- start start 2024-09-10T10:09:15.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.110+0000 7f96bcbe6740 1 Processor -- start 2024-09-10T10:09:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.111+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71baf0 0x557c6b723ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.111+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71c410 0x557c6b724020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 0x557c6b724560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557c6b7298d0 con 0x557c6b71baf0 2024-09-10T10:09:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557c6b609290 con 0x557c6b71c410 2024-09-10T10:09:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71c410 0x557c6b724020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71c410 0x557c6b724020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:33884/0 (socket says 172.21.15.45:33884) 2024-09-10T10:09:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 -- 172.21.15.45:0/2673699220 learned_addr learned my addr 172.21.15.45:0/2673699220 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x557c6b729a50 con 0x557c6b726cb0 2024-09-10T10:09:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96b3fff640 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 0x557c6b724560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 -- 172.21.15.45:0/2673699220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 msgr2=0x557c6b724560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 0x557c6b724560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 -- 172.21.15.45:0/2673699220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71baf0 msgr2=0x557c6b723ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71baf0 0x557c6b723ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.112+0000 7f96abfff640 1 -- 172.21.15.45:0/2673699220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6b393210 con 0x557c6b71c410 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.113+0000 7f96bcbe6740 10 client.0 _ll_register_callbacks cb 0x557c6b63a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.113+0000 7f96abfff640 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71c410 0x557c6b724020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f969c0bbe20 tx=0x7f969c0bbfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=460b0fc4e845ac64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.113+0000 7f9692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:09:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.113+0000 7f9692ffd640 1 -- 172.21.15.45:0/2673699220 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c031e50 con 0x557c6b71c410 2024-09-10T10:09:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.113+0000 7f9692ffd640 1 -- 172.21.15.45:0/2673699220 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f969c0b8790 con 0x557c6b71c410 2024-09-10T10:09:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.113+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96ac010e60 con 0x557c6b71c410 2024-09-10T10:09:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.118+0000 7f9692ffd640 1 -- 172.21.15.45:0/2673699220 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f969c0bc820 con 0x557c6b71c410 2024-09-10T10:09:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.118+0000 7f96bcbe6740 10 client.25163 fetch_fsmap learned FSMap version 21 2024-09-10T10:09:15.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.118+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=21}) -- 0x557c6b45f620 con 0x557c6b71c410 2024-09-10T10:09:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.119+0000 7f9692ffd640 1 -- 172.21.15.45:0/2673699220 <== mon.2 v2:172.21.15.47:3300/0 4 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f969c0c0120 con 0x557c6b71c410 2024-09-10T10:09:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.120+0000 7f96bcbe6740 10 client.25163 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:09:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.120+0000 7f96bcbe6740 10 client.25163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:09:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.120+0000 7f96bcbe6740 20 client.25163 populate_metadata read hostname 'smithi045' 2024-09-10T10:09:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.120+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b736990 0x557c6b738e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.120+0000 7f96bcbe6740 4 client.25163 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:09:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.120+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f96ac010e60 con 0x557c6b736990 2024-09-10T10:09:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.123+0000 7f96b37fe640 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b736990 0x557c6b738e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.127+0000 7f9692ffd640 10 client.25163 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:15.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.127+0000 7f96b37fe640 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b736990 0x557c6b738e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8afeab23ff333e86 server_cookie=d9a5591406646b85 in_seq=0 out_seq=0 2024-09-10T10:09:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.129+0000 7f9692ffd640 1 -- 172.21.15.45:0/2673699220 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f96ac010e60 con 0x557c6b736990 2024-09-10T10:09:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.129+0000 7f9692ffd640 10 client.25163 handle_command_reply: tid=0 2024-09-10T10:09:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.129+0000 7f96bcbe6740 1 client.25163 shutdown 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71c410 msgr2=0x557c6b724020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71c410 0x557c6b724020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f969c0bbe20 tx=0x7f969c0bbfb0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 shutdown_connections 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b736990 0x557c6b738e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 0x557c6b724560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71c410 0x557c6b724020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/2673699220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71baf0 0x557c6b723ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.132+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 >> 172.21.15.45:0/2673699220 conn(0x557c6b718060 msgr2=0x557c6b71dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.134+0000 7f9692ffd640 0 client.25163 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.134+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 shutdown_connections 2024-09-10T10:09:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.135+0000 7f96bcbe6740 1 -- 172.21.15.45:0/2673699220 wait complete. 2024-09-10T10:09:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.135+0000 7f96bcbe6740 20 client.25163 trim_cache size 0 max 16384 2024-09-10T10:09:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.199+0000 7f96bcbe6740 1 Processor -- start 2024-09-10T10:09:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.200+0000 7f96bcbe6740 1 -- start start 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.201+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71baf0 0x557c6b724390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.201+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 0x557c6b73e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b726cb0 0x557c6b73eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b1cb0 con 0x557c6b71c410 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b4550 con 0x557c6b726cb0 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b2340 con 0x557c6b71baf0 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71baf0 0x557c6b724390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71baf0 0x557c6b724390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:55834/0 (socket says 172.21.15.45:55834) 2024-09-10T10:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96b37fe640 1 -- 172.21.15.45:0/4798601 learned_addr learned my addr 172.21.15.45:0/4798601 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96abfff640 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 0x557c6b73e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.202+0000 7f96b3fff640 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b726cb0 0x557c6b73eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.203+0000 7f96b37fe640 1 -- 172.21.15.45:0/4798601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b726cb0 msgr2=0x557c6b73eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.203+0000 7f96b37fe640 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b726cb0 0x557c6b73eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.203+0000 7f96b37fe640 1 -- 172.21.15.45:0/4798601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 msgr2=0x557c6b73e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.203+0000 7f96b37fe640 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 0x557c6b73e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.203+0000 7f96b37fe640 1 -- 172.21.15.45:0/4798601 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6b71f6c0 con 0x557c6b71baf0 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.204+0000 7f96b37fe640 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71baf0 0x557c6b724390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009460 tx=0x7f96a4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e81180c889f993a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.208+0000 7f96a97fa640 1 -- 172.21.15.45:0/4798601 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4005020 con 0x557c6b71baf0 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.208+0000 7f96a97fa640 1 -- 172.21.15.45:0/4798601 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a40038d0 con 0x557c6b71baf0 2024-09-10T10:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.208+0000 7f96a97fa640 1 -- 172.21.15.45:0/4798601 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4004c40 con 0x557c6b71baf0 2024-09-10T10:09:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.212+0000 7f96bcbe6740 1 -- 172.21.15.45:0/4798601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71baf0 msgr2=0x557c6b724390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:15.213 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.212+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71baf0 0x557c6b724390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009460 tx=0x7f96a4017000 comp rx=0 tx=0).stop 2024-09-10T10:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.212+0000 7f96bcbe6740 1 -- 172.21.15.45:0/4798601 shutdown_connections 2024-09-10T10:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.214+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b726cb0 0x557c6b73eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.215+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 0x557c6b73e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.215+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/4798601 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b71baf0 0x557c6b724390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.215+0000 7f96bcbe6740 1 -- 172.21.15.45:0/4798601 >> 172.21.15.45:0/4798601 conn(0x557c6b718060 msgr2=0x557c6b722bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.215+0000 7f96bcbe6740 1 -- 172.21.15.45:0/4798601 shutdown_connections 2024-09-10T10:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.215+0000 7f96bcbe6740 1 -- 172.21.15.45:0/4798601 wait complete. 2024-09-10T10:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.216+0000 7f96bcbe6740 1 -- start start 2024-09-10T10:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.217+0000 7f96bcbe6740 1 Processor -- start 2024-09-10T10:09:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.219+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b72efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.219+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 0x557c6b72f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96bcbe6740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 0x557c6b720440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b2a10 con 0x557c6b71c410 2024-09-10T10:09:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b3e80 con 0x557c6b71baf0 2024-09-10T10:09:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96bcbe6740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41b30e0 con 0x557c6b726cb0 2024-09-10T10:09:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96bcbe6740 10 client.0 _ll_register_callbacks cb 0x7f969c009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:09:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b72efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b72efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:33904/0 (socket says 172.21.15.45:33904) 2024-09-10T10:09:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.220+0000 7f96b37fe640 1 -- 172.21.15.45:0/3504309388 learned_addr learned my addr 172.21.15.45:0/3504309388 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.221+0000 7f96b37fe640 1 -- 172.21.15.45:0/3504309388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 msgr2=0x557c6b720440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:09:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.221+0000 7f96b37fe640 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 0x557c6b720440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.221+0000 7f96b37fe640 1 -- 172.21.15.45:0/3504309388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 msgr2=0x557c6b72f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.221+0000 7f96b37fe640 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 0x557c6b72f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.221+0000 7f96b37fe640 1 -- 172.21.15.45:0/3504309388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6b71c240 con 0x557c6b71baf0 2024-09-10T10:09:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.222+0000 7f96b37fe640 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b72efd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009dc0 tx=0x7f96a4009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8b60e566dfa2958 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.222+0000 7f9692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:09:15.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.224+0000 7f9692ffd640 1 -- 172.21.15.45:0/3504309388 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4003c80 con 0x557c6b71baf0 2024-09-10T10:09:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.224+0000 7f9692ffd640 1 -- 172.21.15.45:0/3504309388 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7f96a4003e20 con 0x557c6b71baf0 2024-09-10T10:09:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.224+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a40038d0 con 0x557c6b71baf0 2024-09-10T10:09:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.226+0000 7f9692ffd640 1 -- 172.21.15.45:0/3504309388 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f96a4041050 con 0x557c6b71baf0 2024-09-10T10:09:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.227+0000 7f96bcbe6740 10 client.25169 fetch_fsmap learned FSMap version 21 2024-09-10T10:09:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.227+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x557c6b7a1100 con 0x557c6b71baf0 2024-09-10T10:09:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.228+0000 7f9692ffd640 1 -- 172.21.15.45:0/3504309388 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96a40038d0 con 0x557c6b71baf0 2024-09-10T10:09:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.229+0000 7f9692ffd640 1 -- 172.21.15.45:0/3504309388 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f96a401f430 con 0x557c6b71baf0 2024-09-10T10:09:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.229+0000 7f96bcbe6740 10 client.25169 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:09:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.229+0000 7f96bcbe6740 10 client.25169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:09:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.229+0000 7f96bcbe6740 20 client.25169 populate_metadata read hostname 'smithi045' 2024-09-10T10:09:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.229+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b7a17d0 0x557c6b7a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.229+0000 7f96bcbe6740 4 client.25169 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:09:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.229+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c6b68da20 con 0x557c6b7a17d0 2024-09-10T10:09:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.236+0000 7f96abfff640 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b7a17d0 0x557c6b7a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.262+0000 7f96abfff640 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b7a17d0 0x557c6b7a3be0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa27e1e598227891 server_cookie=a58a550da3888ae in_seq=0 out_seq=0 2024-09-10T10:09:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:15.262+0000 7f9692ffd640 10 client.25169 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:15.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:15 smithi162 ceph-mon[34963]: pgmap v185: 97 pgs: 97 active+clean; 53 GiB data, 303 GiB used, 770 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 912 MiB/s wr, 4.30k op/s 2024-09-10T10:09:15.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:14 smithi045 ceph-mon[31114]: pgmap v185: 97 pgs: 97 active+clean; 53 GiB data, 303 GiB used, 770 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 912 MiB/s wr, 4.30k op/s 2024-09-10T10:09:15.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:14 smithi047 ceph-mon[35244]: pgmap v185: 97 pgs: 97 active+clean; 53 GiB data, 303 GiB used, 770 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 912 MiB/s wr, 4.30k op/s 2024-09-10T10:09:17.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:17 smithi162 ceph-mon[34963]: pgmap v186: 97 pgs: 97 active+clean; 53 GiB data, 304 GiB used, 769 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 761 MiB/s wr, 4.37k op/s 2024-09-10T10:09:17.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:17 smithi045 ceph-mon[31114]: pgmap v186: 97 pgs: 97 active+clean; 53 GiB data, 304 GiB used, 769 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 761 MiB/s wr, 4.37k op/s 2024-09-10T10:09:17.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:17 smithi047 ceph-mon[35244]: pgmap v186: 97 pgs: 97 active+clean; 53 GiB data, 304 GiB used, 769 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 761 MiB/s wr, 4.37k op/s 2024-09-10T10:09:19.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:19 smithi162 ceph-mon[34963]: pgmap v187: 97 pgs: 97 active+clean; 54 GiB data, 308 GiB used, 765 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 834 MiB/s wr, 4.95k op/s 2024-09-10T10:09:19.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:19 smithi045 ceph-mon[31114]: pgmap v187: 97 pgs: 97 active+clean; 54 GiB data, 308 GiB used, 765 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 834 MiB/s wr, 4.95k op/s 2024-09-10T10:09:19.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:19 smithi047 ceph-mon[35244]: pgmap v187: 97 pgs: 97 active+clean; 54 GiB data, 308 GiB used, 765 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 834 MiB/s wr, 4.95k op/s 2024-09-10T10:09:21.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:21 smithi162 ceph-mon[34963]: pgmap v188: 97 pgs: 97 active+clean; 56 GiB data, 314 GiB used, 759 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 822 MiB/s wr, 4.74k op/s 2024-09-10T10:09:21.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:21 smithi045 ceph-mon[31114]: pgmap v188: 97 pgs: 97 active+clean; 56 GiB data, 314 GiB used, 759 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 822 MiB/s wr, 4.74k op/s 2024-09-10T10:09:21.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:21 smithi047 ceph-mon[35244]: pgmap v188: 97 pgs: 97 active+clean; 56 GiB data, 314 GiB used, 759 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 822 MiB/s wr, 4.74k op/s 2024-09-10T10:09:23.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:23 smithi047 ceph-mon[35244]: pgmap v189: 97 pgs: 97 active+clean; 57 GiB data, 316 GiB used, 757 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 734 MiB/s wr, 3.68k op/s 2024-09-10T10:09:23.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:23 smithi162 ceph-mon[34963]: pgmap v189: 97 pgs: 97 active+clean; 57 GiB data, 316 GiB used, 757 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 734 MiB/s wr, 3.68k op/s 2024-09-10T10:09:23.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:23 smithi045 ceph-mon[31114]: pgmap v189: 97 pgs: 97 active+clean; 57 GiB data, 316 GiB used, 757 GiB / 1.0 TiB avail; 4.7 MiB/s rd, 734 MiB/s wr, 3.68k op/s 2024-09-10T10:09:25.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:25 smithi162 ceph-mon[34963]: pgmap v190: 97 pgs: 97 active+clean; 58 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 5.6 MiB/s rd, 936 MiB/s wr, 3.64k op/s 2024-09-10T10:09:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:25 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:09:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:25 smithi045 ceph-mon[31114]: pgmap v190: 97 pgs: 97 active+clean; 58 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 5.6 MiB/s rd, 936 MiB/s wr, 3.64k op/s 2024-09-10T10:09:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:25 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:09:25.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:25 smithi047 ceph-mon[35244]: pgmap v190: 97 pgs: 97 active+clean; 58 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 5.6 MiB/s rd, 936 MiB/s wr, 3.64k op/s 2024-09-10T10:09:25.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:25 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:09:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:27 smithi162 ceph-mon[34963]: pgmap v191: 97 pgs: 97 active+clean; 58 GiB data, 324 GiB used, 749 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 774 MiB/s wr, 2.21k op/s 2024-09-10T10:09:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:09:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:09:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:09:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:27 smithi045 ceph-mon[31114]: pgmap v191: 97 pgs: 97 active+clean; 58 GiB data, 324 GiB used, 749 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 774 MiB/s wr, 2.21k op/s 2024-09-10T10:09:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:09:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:09:27.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:09:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:27 smithi047 ceph-mon[35244]: pgmap v191: 97 pgs: 97 active+clean; 58 GiB data, 324 GiB used, 749 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 774 MiB/s wr, 2.21k op/s 2024-09-10T10:09:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:09:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:09:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.859+0000 7f9692ffd640 1 -- 172.21.15.45:0/3504309388 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c6b68da20 con 0x557c6b7a17d0 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.859+0000 7f9692ffd640 10 client.25169 handle_command_reply: tid=0 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.859+0000 7f96bcbe6740 1 client.25169 shutdown 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 msgr2=0x557c6b72efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b72efd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009dc0 tx=0x7f96a4009df0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 shutdown_connections 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x557c6b7a17d0 0x557c6b7a3be0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x557c6b726cb0 0x557c6b720440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6b71c410 0x557c6b72f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 --2- 172.21.15.45:0/3504309388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557c6b71baf0 0x557c6b72efd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.862+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 >> 172.21.15.45:0/3504309388 conn(0x557c6b718060 msgr2=0x557c6b71d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.863+0000 7f9692ffd640 0 client.25169 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.864+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 shutdown_connections 2024-09-10T10:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.864+0000 7f96bcbe6740 1 -- 172.21.15.45:0/3504309388 wait complete. 2024-09-10T10:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.864+0000 7f96bcbe6740 20 client.25169 trim_cache size 0 max 16384 2024-09-10T10:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:09:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f96a00a2f00 msgr2=0x7f96a00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f96a00a2f00 0x7f96a00a53c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x557c6b718ec0 tx=0x7f96a4005f90 comp rx=0 tx=0).stop 2024-09-10T10:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b41009e0 msgr2=0x7f96b41a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b41009e0 0x7f96b41a4bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f969c004770 tx=0x7f969c003c10 comp rx=0 tx=0).stop 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 shutdown_connections 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96b4106be0 0x7f96b419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f96b41009e0 0x7f96b41a4bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f96a00a2f00 0x7f96a00a53c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 --2- 172.21.15.45:0/1926833687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b40fff60 0x7f96b41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 >> 172.21.15.45:0/1926833687 conn(0x7f96b40770f0 msgr2=0x7f96b4102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 shutdown_connections 2024-09-10T10:09:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:28.865+0000 7f96b9e4c640 1 -- 172.21.15.45:0/1926833687 wait complete. 2024-09-10T10:09:29.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:29 smithi162 ceph-mon[34963]: pgmap v192: 97 pgs: 97 active+clean; 59 GiB data, 326 GiB used, 747 GiB / 1.0 TiB avail; 6.0 MiB/s rd, 863 MiB/s wr, 2.20k op/s 2024-09-10T10:09:29.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:29 smithi045 ceph-mon[31114]: pgmap v192: 97 pgs: 97 active+clean; 59 GiB data, 326 GiB used, 747 GiB / 1.0 TiB avail; 6.0 MiB/s rd, 863 MiB/s wr, 2.20k op/s 2024-09-10T10:09:29.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:29 smithi047 ceph-mon[35244]: pgmap v192: 97 pgs: 97 active+clean; 59 GiB data, 326 GiB used, 747 GiB / 1.0 TiB avail; 6.0 MiB/s rd, 863 MiB/s wr, 2.20k op/s 2024-09-10T10:09:31.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:31 smithi162 ceph-mon[34963]: pgmap v193: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 6.4 MiB/s rd, 882 MiB/s wr, 1.98k op/s 2024-09-10T10:09:31.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:31 smithi045 ceph-mon[31114]: pgmap v193: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 6.4 MiB/s rd, 882 MiB/s wr, 1.98k op/s 2024-09-10T10:09:31.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:31 smithi047 ceph-mon[35244]: pgmap v193: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 6.4 MiB/s rd, 882 MiB/s wr, 1.98k op/s 2024-09-10T10:09:33.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:33 smithi162 ceph-mon[34963]: pgmap v194: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 799 MiB/s wr, 1.59k op/s 2024-09-10T10:09:33.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:33 smithi047 ceph-mon[35244]: pgmap v194: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 799 MiB/s wr, 1.59k op/s 2024-09-10T10:09:33.910 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:33 smithi045 ceph-mon[31114]: pgmap v194: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 799 MiB/s wr, 1.59k op/s 2024-09-10T10:09:35.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:35 smithi162 ceph-mon[34963]: pgmap v195: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 5.9 MiB/s rd, 938 MiB/s wr, 1.98k op/s 2024-09-10T10:09:35.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:35 smithi045 ceph-mon[31114]: pgmap v195: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 5.9 MiB/s rd, 938 MiB/s wr, 1.98k op/s 2024-09-10T10:09:35.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:35 smithi047 ceph-mon[35244]: pgmap v195: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 5.9 MiB/s rd, 938 MiB/s wr, 1.98k op/s 2024-09-10T10:09:37.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:37 smithi162 ceph-mon[34963]: pgmap v196: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 782 MiB/s wr, 1.87k op/s 2024-09-10T10:09:37.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:37 smithi045 ceph-mon[31114]: pgmap v196: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 782 MiB/s wr, 1.87k op/s 2024-09-10T10:09:37.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:37 smithi047 ceph-mon[35244]: pgmap v196: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 782 MiB/s wr, 1.87k op/s 2024-09-10T10:09:39.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:39 smithi162 ceph-mon[34963]: pgmap v197: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 859 MiB/s wr, 2.32k op/s 2024-09-10T10:09:39.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:39 smithi045 ceph-mon[31114]: pgmap v197: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 859 MiB/s wr, 2.32k op/s 2024-09-10T10:09:39.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:39 smithi047 ceph-mon[35244]: pgmap v197: 97 pgs: 97 active+clean; 60 GiB data, 335 GiB used, 738 GiB / 1.0 TiB avail; 4.3 MiB/s rd, 859 MiB/s wr, 2.32k op/s 2024-09-10T10:09:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.135+0000 7fe27728f640 1 -- 172.21.15.45:0/3812254793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 msgr2=0x7fe270110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.135+0000 7fe27728f640 1 --2- 172.21.15.45:0/3812254793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe270110250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009900 tx=0x7fe26c02a440 comp rx=0 tx=0).stop 2024-09-10T10:09:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.136+0000 7fe27728f640 1 -- 172.21.15.45:0/3812254793 shutdown_connections 2024-09-10T10:09:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.136+0000 7fe27728f640 1 --2- 172.21.15.45:0/3812254793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27011b140 0x7fe27011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.136+0000 7fe27728f640 1 --2- 172.21.15.45:0/3812254793 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2701095d0 0x7fe270109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.136+0000 7fe27728f640 1 --2- 172.21.15.45:0/3812254793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe270110250 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.136+0000 7fe27728f640 1 -- 172.21.15.45:0/3812254793 >> 172.21.15.45:0/3812254793 conn(0x7fe27006cbf0 msgr2=0x7fe27006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.139+0000 7fe27728f640 1 -- 172.21.15.45:0/3812254793 shutdown_connections 2024-09-10T10:09:42.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:41 smithi162 ceph-mon[34963]: pgmap v198: 97 pgs: 97 active+clean; 60 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 860 MiB/s wr, 2.37k op/s 2024-09-10T10:09:42.158 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:41 smithi045 ceph-mon[31114]: pgmap v198: 97 pgs: 97 active+clean; 60 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 860 MiB/s wr, 2.37k op/s 2024-09-10T10:09:42.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:41 smithi047 ceph-mon[35244]: pgmap v198: 97 pgs: 97 active+clean; 60 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 860 MiB/s wr, 2.37k op/s 2024-09-10T10:09:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.146+0000 7fe27728f640 1 -- 172.21.15.45:0/3812254793 wait complete. 2024-09-10T10:09:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.147+0000 7fe27728f640 1 Processor -- start 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.148+0000 7fe27728f640 1 -- start start 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.148+0000 7fe27728f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe2701187e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.148+0000 7fe27728f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2701095d0 0x7fe270111860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.148+0000 7fe27728f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27011b140 0x7fe270111da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.148+0000 7fe27728f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe270120bd0 con 0x7fe2701091f0 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.148+0000 7fe27728f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe270120a50 con 0x7fe27011b140 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.148+0000 7fe27728f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe270120d50 con 0x7fe2701095d0 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.152+0000 7fe27628d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe2701187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.152+0000 7fe27628d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe2701187e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33892/0 (socket says 172.21.15.45:33892) 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.152+0000 7fe27628d640 1 -- 172.21.15.45:0/1857761158 learned_addr learned my addr 172.21.15.45:0/1857761158 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.152+0000 7fe275a8c640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2701095d0 0x7fe270111860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.152+0000 7fe276a8e640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27011b140 0x7fe270111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.157+0000 7fe27628d640 1 -- 172.21.15.45:0/1857761158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2701095d0 msgr2=0x7fe270111860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.157+0000 7fe27628d640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2701095d0 0x7fe270111860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.157+0000 7fe27628d640 1 -- 172.21.15.45:0/1857761158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27011b140 msgr2=0x7fe270111da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.157+0000 7fe27628d640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27011b140 0x7fe270111da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.157+0000 7fe27628d640 1 -- 172.21.15.45:0/1857761158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe270112440 con 0x7fe2701091f0 2024-09-10T10:09:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.157+0000 7fe27628d640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe2701187e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe26c034040 tx=0x7fe26c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:42.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.166+0000 7fe2677fe640 1 -- 172.21.15.45:0/1857761158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c041020 con 0x7fe2701091f0 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.166+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe270116300 con 0x7fe2701091f0 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.166+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2701167f0 con 0x7fe2701091f0 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.173+0000 7fe27b28b740 1 Processor -- start 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.174+0000 7fe27b28b740 1 -- start start 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05ab5e0 0x5556a05b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05b5e80 0x5556a05b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe27b28b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5556a050f2f0 con 0x5556a05aacc0 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe27b28b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5556a04b7270 con 0x5556a05ab5e0 2024-09-10T10:09:42.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe27b28b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5556a050b2b0 con 0x5556a05b5e80 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe2677fe640 1 -- 172.21.15.45:0/1857761158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c0319c0 con 0x7fe2701091f0 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.176+0000 7fe2677fe640 1 -- 172.21.15.45:0/1857761158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c036040 con 0x7fe2701091f0 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.177+0000 7fe27628d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05ab5e0 0x5556a05b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.177+0000 7fe27628d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05ab5e0 0x5556a05b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:47724/0 (socket says 172.21.15.45:47724) 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.177+0000 7fe27628d640 1 -- 172.21.15.45:0/4216740098 learned_addr learned my addr 172.21.15.45:0/4216740098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.178+0000 7fe275a8c640 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05b5e80 0x5556a05b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.179+0000 7fe27628d640 1 -- 172.21.15.45:0/4216740098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05b5e80 msgr2=0x5556a05b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.179+0000 7fe27628d640 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05b5e80 0x5556a05b8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.179+0000 7fe27628d640 1 -- 172.21.15.45:0/4216740098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 msgr2=0x5556a05ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.179+0000 7fe27628d640 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.179+0000 7fe27628d640 1 -- 172.21.15.45:0/4216740098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a04b6340 con 0x5556a05ab5e0 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.180+0000 7fe27628d640 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05ab5e0 0x5556a05b5940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe26c005ab0 tx=0x7fe26c040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c139839e4c47e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.181+0000 7fe264bfe640 1 -- 172.21.15.45:0/4216740098 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c060070 con 0x5556a05ab5e0 2024-09-10T10:09:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.181+0000 7fe264bfe640 1 -- 172.21.15.45:0/4216740098 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c004c10 con 0x5556a05ab5e0 2024-09-10T10:09:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.183+0000 7fe27b28b740 1 -- 172.21.15.45:0/4216740098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05ab5e0 msgr2=0x5556a05b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.183+0000 7fe27b28b740 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05ab5e0 0x5556a05b5940 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe26c005ab0 tx=0x7fe26c040000 comp rx=0 tx=0).stop 2024-09-10T10:09:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.189+0000 7fe2677fe640 1 -- 172.21.15.45:0/1857761158 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe26c044020 con 0x7fe2701091f0 2024-09-10T10:09:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.190+0000 7fe2677fe640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe25c0a2f00 0x7fe25c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.191+0000 7fe276a8e640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe25c0a2f00 0x7fe25c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.191+0000 7fe2677fe640 1 -- 172.21.15.45:0/1857761158 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7fe26c0f4270 con 0x7fe2701091f0 2024-09-10T10:09:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.192+0000 7fe27b28b740 1 -- 172.21.15.45:0/4216740098 shutdown_connections 2024-09-10T10:09:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.192+0000 7fe27b28b740 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05b5e80 0x5556a05b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.192+0000 7fe27b28b740 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05ab5e0 0x5556a05b5940 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.192+0000 7fe27b28b740 1 --2- 172.21.15.45:0/4216740098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.192+0000 7fe27b28b740 1 -- 172.21.15.45:0/4216740098 >> 172.21.15.45:0/4216740098 conn(0x5556a05a7230 msgr2=0x5556a05a7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:42.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.193+0000 7fe27b28b740 1 -- 172.21.15.45:0/4216740098 shutdown_connections 2024-09-10T10:09:42.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.193+0000 7fe27b28b740 1 -- 172.21.15.45:0/4216740098 wait complete. 2024-09-10T10:09:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.193+0000 7fe27b28b740 1 -- start start 2024-09-10T10:09:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.194+0000 7fe27b28b740 1 Processor -- start 2024-09-10T10:09:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.198+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.200+0000 7fe27628d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.200+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.201+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.201+0000 7fe27b28b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5556a05da820 con 0x5556a05aacc0 2024-09-10T10:09:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.201+0000 7fe27b28b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5556a0498290 con 0x5556a05b5e80 2024-09-10T10:09:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.201+0000 7fe27b28b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5556a05da9a0 con 0x5556a05ab5e0 2024-09-10T10:09:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.201+0000 7fe27b28b740 10 client.0 _ll_register_callbacks cb 0x5556a04c9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:09:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.202+0000 7fe275a8c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.202+0000 7fe275a8c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:49810/0 (socket says 172.21.15.45:49810) 2024-09-10T10:09:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.202+0000 7fe275a8c640 1 -- 172.21.15.45:0/681103234 learned_addr learned my addr 172.21.15.45:0/681103234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.203+0000 7fe276a8e640 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.206+0000 7fe275a8c640 1 -- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 msgr2=0x5556a05b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.206+0000 7fe275a8c640 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.206+0000 7fe275a8c640 1 -- 172.21.15.45:0/681103234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 msgr2=0x5556a05b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.206+0000 7fe275a8c640 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.206+0000 7fe275a8c640 1 -- 172.21.15.45:0/681103234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a0222210 con 0x5556a05ab5e0 2024-09-10T10:09:42.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.207+0000 7fe276a8e640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe25c0a2f00 0x7fe25c0a53c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe268008b90 tx=0x7fe268008260 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.215+0000 7fe275a8c640 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b2bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe26000f4c0 tx=0x7fe26000f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429bdd800367d1ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.216+0000 7fe259ffb640 10 client.0 ms_handle_connect on v2:172.21.15.162:3300/0 2024-09-10T10:09:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.216+0000 7fe259ffb640 1 -- 172.21.15.45:0/681103234 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260013850 con 0x5556a05ab5e0 2024-09-10T10:09:42.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.216+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26c004c10 con 0x5556a05ab5e0 2024-09-10T10:09:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.217+0000 7fe259ffb640 1 -- 172.21.15.45:0/681103234 <== mon.1 v2:172.21.15.162:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7fe2600139f0 con 0x5556a05ab5e0 2024-09-10T10:09:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.217+0000 7fe259ffb640 1 -- 172.21.15.45:0/681103234 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe260013bf0 con 0x5556a05ab5e0 2024-09-10T10:09:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.218+0000 7fe27b28b740 10 client.25096 fetch_fsmap learned FSMap version 21 2024-09-10T10:09:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.218+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5556a02ee620 con 0x5556a05ab5e0 2024-09-10T10:09:42.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.222+0000 7fe259ffb640 1 -- 172.21.15.45:0/681103234 <== mon.1 v2:172.21.15.162:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe260013d90 con 0x5556a05ab5e0 2024-09-10T10:09:42.225 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.222+0000 7fe259ffb640 1 -- 172.21.15.45:0/681103234 <== mon.1 v2:172.21.15.162:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe260011620 con 0x5556a05ab5e0 2024-09-10T10:09:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.224+0000 7fe27b28b740 10 client.25096 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:09:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.224+0000 7fe27b28b740 10 client.25096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:09:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.224+0000 7fe27b28b740 20 client.25096 populate_metadata read hostname 'smithi045' 2024-09-10T10:09:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.224+0000 7fe27b28b740 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a05e7b90 0x5556a05e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.224+0000 7fe27b28b740 4 client.25096 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:09:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.224+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe26c004c10 con 0x5556a05e7b90 2024-09-10T10:09:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.227+0000 7fe27628d640 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a05e7b90 0x5556a05e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.237+0000 7fe27628d640 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a05e7b90 0x5556a05e9fb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5672a3dda6ff5772 server_cookie=994ab8c30c279a26 in_seq=0 out_seq=0 2024-09-10T10:09:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.237+0000 7fe259ffb640 10 client.25096 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.243+0000 7fe259ffb640 1 -- 172.21.15.45:0/681103234 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe26c004c10 con 0x5556a05e7b90 2024-09-10T10:09:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.243+0000 7fe259ffb640 10 client.25096 handle_command_reply: tid=0 2024-09-10T10:09:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.243+0000 7fe27b28b740 1 client.25096 shutdown 2024-09-10T10:09:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 msgr2=0x5556a05b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b2bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe26000f4c0 tx=0x7fe26000f980 comp rx=0 tx=0).stop 2024-09-10T10:09:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 shutdown_connections 2024-09-10T10:09:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a05e7b90 0x5556a05e9fb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b2bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 --2- 172.21.15.45:0/681103234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.246+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 >> 172.21.15.45:0/681103234 conn(0x5556a05a7230 msgr2=0x5556a05acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:42.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.247+0000 7fe259ffb640 0 client.25096 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.247+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 shutdown_connections 2024-09-10T10:09:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.247+0000 7fe27b28b740 1 -- 172.21.15.45:0/681103234 wait complete. 2024-09-10T10:09:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.247+0000 7fe27b28b740 20 client.25096 trim_cache size 0 max 16384 2024-09-10T10:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.294+0000 7fe27b28b740 1 Processor -- start 2024-09-10T10:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.294+0000 7fe27b28b740 1 -- start start 2024-09-10T10:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27b28b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe25400c810 con 0x5556a05aacc0 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27b28b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe25400f0b0 con 0x5556a05b5e80 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27b28b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe25400cea0 con 0x5556a05ab5e0 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27628d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27628d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33910/0 (socket says 172.21.15.45:33910) 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.295+0000 7fe27628d640 1 -- 172.21.15.45:0/2283716728 learned_addr learned my addr 172.21.15.45:0/2283716728 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.296+0000 7fe27628d640 1 -- 172.21.15.45:0/2283716728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 msgr2=0x5556a05ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.296+0000 7fe27628d640 1 --2- 172.21.15.45:0/2283716728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.296+0000 7fe27628d640 1 -- 172.21.15.45:0/2283716728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 msgr2=0x5556a05efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.296+0000 7fe27628d640 1 --2- 172.21.15.45:0/2283716728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.296+0000 7fe27628d640 1 -- 172.21.15.45:0/2283716728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556a05b1100 con 0x5556a05aacc0 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.296+0000 7fe27628d640 1 --2- 172.21.15.45:0/2283716728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b1a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0c0830 tx=0x7fe26c02a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934a7bf6b0f95b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.297+0000 7fe25bfff640 1 -- 172.21.15.45:0/2283716728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0c0200 con 0x5556a05aacc0 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.297+0000 7fe25bfff640 1 -- 172.21.15.45:0/2283716728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c0c08a0 con 0x5556a05aacc0 2024-09-10T10:09:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.297+0000 7fe25bfff640 1 -- 172.21.15.45:0/2283716728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c03cac0 con 0x5556a05aacc0 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.298+0000 7fe27b28b740 1 -- 172.21.15.45:0/2283716728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 msgr2=0x5556a05b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.298+0000 7fe27b28b740 1 --2- 172.21.15.45:0/2283716728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b1a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0c0830 tx=0x7fe26c02a440 comp rx=0 tx=0).stop 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.299+0000 7fe27b28b740 1 -- 172.21.15.45:0/2283716728 shutdown_connections 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.299+0000 7fe27b28b740 1 --2- 172.21.15.45:0/2283716728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.299+0000 7fe27b28b740 1 --2- 172.21.15.45:0/2283716728 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.299+0000 7fe27b28b740 1 --2- 172.21.15.45:0/2283716728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05b1a60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.299+0000 7fe27b28b740 1 -- 172.21.15.45:0/2283716728 >> 172.21.15.45:0/2283716728 conn(0x5556a05a7230 msgr2=0x5556a05b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.299+0000 7fe27b28b740 1 -- 172.21.15.45:0/2283716728 shutdown_connections 2024-09-10T10:09:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.299+0000 7fe27b28b740 1 -- 172.21.15.45:0/2283716728 wait complete. 2024-09-10T10:09:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.300+0000 7fe27b28b740 1 -- start start 2024-09-10T10:09:42.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.300+0000 7fe27b28b740 1 Processor -- start 2024-09-10T10:09:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.300+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05e0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.301+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.301+0000 7fe27b28b740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.301+0000 7fe27b28b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe25400d570 con 0x5556a05aacc0 2024-09-10T10:09:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.301+0000 7fe27b28b740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe25400e9e0 con 0x5556a05b5e80 2024-09-10T10:09:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.301+0000 7fe27b28b740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe25400dc40 con 0x5556a05ab5e0 2024-09-10T10:09:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.301+0000 7fe27b28b740 10 client.0 _ll_register_callbacks cb 0x7fe26000a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:09:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.301+0000 7fe27628d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.302+0000 7fe27628d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05e0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33924/0 (socket says 172.21.15.45:33924) 2024-09-10T10:09:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.302+0000 7fe27628d640 1 -- 172.21.15.45:0/3187193338 learned_addr learned my addr 172.21.15.45:0/3187193338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:42.305 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.302+0000 7fe276a8e640 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe27628d640 1 -- 172.21.15.45:0/3187193338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 msgr2=0x5556a05b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe27628d640 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe27628d640 1 -- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 msgr2=0x5556a05b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe27628d640 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe27628d640 1 -- 172.21.15.45:0/3187193338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556a05b7860 con 0x5556a05aacc0 2024-09-10T10:09:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe276a8e640 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:09:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe27628d640 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05e0300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0bcd00 tx=0x7fe26c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e266136bb0daaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.303+0000 7fe259ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:09:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.304+0000 7fe259ffb640 1 -- 172.21.15.45:0/3187193338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0d8020 con 0x5556a05aacc0 2024-09-10T10:09:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.304+0000 7fe259ffb640 1 -- 172.21.15.45:0/3187193338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7fe26c0c7070 con 0x5556a05aacc0 2024-09-10T10:09:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.304+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26c0c08a0 con 0x5556a05aacc0 2024-09-10T10:09:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.305+0000 7fe259ffb640 1 -- 172.21.15.45:0/3187193338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe26c0f4040 con 0x5556a05aacc0 2024-09-10T10:09:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.305+0000 7fe27b28b740 10 client.15519 fetch_fsmap learned FSMap version 21 2024-09-10T10:09:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.305+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5556a0652850 con 0x5556a05aacc0 2024-09-10T10:09:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe259ffb640 1 -- 172.21.15.45:0/3187193338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe26c0c08a0 con 0x5556a05aacc0 2024-09-10T10:09:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe259ffb640 1 -- 172.21.15.45:0/3187193338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fe26c004c10 con 0x5556a05aacc0 2024-09-10T10:09:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe27b28b740 10 client.15519 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:09:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe27b28b740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:09:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe27b28b740 20 client.15519 populate_metadata read hostname 'smithi045' 2024-09-10T10:09:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe27b28b740 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a0652ad0 0x5556a0654ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe27b28b740 4 client.15519 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:09:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.306+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556a051ca20 con 0x5556a0652ad0 2024-09-10T10:09:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.307+0000 7fe275a8c640 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a0652ad0 0x5556a0654ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.311+0000 7fe275a8c640 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a0652ad0 0x5556a0654ec0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7934961972043471 server_cookie=de3580a59c4f9668 in_seq=0 out_seq=0 2024-09-10T10:09:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:42.311+0000 7fe259ffb640 10 client.15519 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:44.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:43 smithi162 ceph-mon[34963]: pgmap v199: 97 pgs: 97 active+clean; 60 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 782 MiB/s wr, 2.20k op/s 2024-09-10T10:09:44.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:43 smithi045 ceph-mon[31114]: pgmap v199: 97 pgs: 97 active+clean; 60 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 782 MiB/s wr, 2.20k op/s 2024-09-10T10:09:44.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:43 smithi047 ceph-mon[35244]: pgmap v199: 97 pgs: 97 active+clean; 60 GiB data, 323 GiB used, 750 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 782 MiB/s wr, 2.20k op/s 2024-09-10T10:09:45.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.252+0000 7fe259ffb640 1 -- 172.21.15.45:0/3187193338 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556a051ca20 con 0x5556a0652ad0 2024-09-10T10:09:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.252+0000 7fe259ffb640 10 client.15519 handle_command_reply: tid=0 2024-09-10T10:09:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.252+0000 7fe27b28b740 1 client.15519 shutdown 2024-09-10T10:09:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 msgr2=0x5556a05e0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05e0300 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0bcd00 tx=0x7fe26c0056d0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 shutdown_connections 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5556a0652ad0 0x5556a0654ec0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5556a05b5e80 0x5556a05b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5556a05ab5e0 0x5556a05b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 --2- 172.21.15.45:0/3187193338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5556a05aacc0 0x5556a05e0300 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 >> 172.21.15.45:0/3187193338 conn(0x5556a05a7230 msgr2=0x5556a05e3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe259ffb640 0 client.15519 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.253+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 shutdown_connections 2024-09-10T10:09:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.254+0000 7fe27b28b740 1 -- 172.21.15.45:0/3187193338 wait complete. 2024-09-10T10:09:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.254+0000 7fe27b28b740 20 client.15519 trim_cache size 0 max 16384 2024-09-10T10:09:45.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:09:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:09:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:09:45.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:09:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.282+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe25c0a2f00 msgr2=0x7fe25c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.282+0000 7fe27728f640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe25c0a2f00 0x7fe25c0a53c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe268008b90 tx=0x7fe268008260 comp rx=0 tx=0).stop 2024-09-10T10:09:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.282+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 msgr2=0x7fe2701187e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:45.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.282+0000 7fe27728f640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe2701187e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe26c034040 tx=0x7fe26c005270 comp rx=0 tx=0).stop 2024-09-10T10:09:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.294+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 shutdown_connections 2024-09-10T10:09:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.294+0000 7fe27728f640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe25c0a2f00 0x7fe25c0a53c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.294+0000 7fe27728f640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe27011b140 0x7fe270111da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.294+0000 7fe27728f640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2701095d0 0x7fe270111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.294+0000 7fe27728f640 1 --2- 172.21.15.45:0/1857761158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2701091f0 0x7fe2701187e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.294+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 >> 172.21.15.45:0/1857761158 conn(0x7fe27006cbf0 msgr2=0x7fe27011b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.295+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 shutdown_connections 2024-09-10T10:09:45.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:45.296+0000 7fe27728f640 1 -- 172.21.15.45:0/1857761158 wait complete. 2024-09-10T10:09:46.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:45 smithi162 ceph-mon[34963]: pgmap v200: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 5.5 MiB/s rd, 927 MiB/s wr, 2.55k op/s 2024-09-10T10:09:46.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:45 smithi045 ceph-mon[31114]: pgmap v200: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 5.5 MiB/s rd, 927 MiB/s wr, 2.55k op/s 2024-09-10T10:09:46.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:45 smithi047 ceph-mon[35244]: pgmap v200: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 5.5 MiB/s rd, 927 MiB/s wr, 2.55k op/s 2024-09-10T10:09:48.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:47 smithi162 ceph-mon[34963]: pgmap v201: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 762 MiB/s wr, 2.11k op/s 2024-09-10T10:09:48.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:47 smithi047 ceph-mon[35244]: pgmap v201: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 762 MiB/s wr, 2.11k op/s 2024-09-10T10:09:48.168 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:47 smithi045 ceph-mon[31114]: pgmap v201: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 762 MiB/s wr, 2.11k op/s 2024-09-10T10:09:50.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:49 smithi047 ceph-mon[35244]: pgmap v202: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 835 MiB/s wr, 2.29k op/s 2024-09-10T10:09:50.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:49 smithi162 ceph-mon[34963]: pgmap v202: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 835 MiB/s wr, 2.29k op/s 2024-09-10T10:09:50.403 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:49 smithi045 ceph-mon[31114]: pgmap v202: 97 pgs: 97 active+clean; 60 GiB data, 327 GiB used, 746 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 835 MiB/s wr, 2.29k op/s 2024-09-10T10:09:52.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:51 smithi162 ceph-mon[34963]: pgmap v203: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 844 MiB/s wr, 2.17k op/s 2024-09-10T10:09:52.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:51 smithi045 ceph-mon[31114]: pgmap v203: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 844 MiB/s wr, 2.17k op/s 2024-09-10T10:09:52.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:51 smithi047 ceph-mon[35244]: pgmap v203: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 5.0 MiB/s rd, 844 MiB/s wr, 2.17k op/s 2024-09-10T10:09:53.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:53 smithi162 ceph-mon[34963]: pgmap v204: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 731 MiB/s wr, 1.84k op/s 2024-09-10T10:09:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:53 smithi045 ceph-mon[31114]: pgmap v204: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 731 MiB/s wr, 1.84k op/s 2024-09-10T10:09:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:53 smithi047 ceph-mon[35244]: pgmap v204: 97 pgs: 97 active+clean; 60 GiB data, 331 GiB used, 742 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 731 MiB/s wr, 1.84k op/s 2024-09-10T10:09:55.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:55 smithi162 ceph-mon[34963]: pgmap v205: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 944 MiB/s wr, 1.99k op/s 2024-09-10T10:09:55.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:55 smithi045 ceph-mon[31114]: pgmap v205: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 944 MiB/s wr, 1.99k op/s 2024-09-10T10:09:55.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:55 smithi047 ceph-mon[35244]: pgmap v205: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 5.3 MiB/s rd, 944 MiB/s wr, 1.99k op/s 2024-09-10T10:09:57.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:57 smithi162 ceph-mon[34963]: pgmap v206: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 790 MiB/s wr, 1.55k op/s 2024-09-10T10:09:57.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:57 smithi045 ceph-mon[31114]: pgmap v206: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 790 MiB/s wr, 1.55k op/s 2024-09-10T10:09:57.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:57 smithi047 ceph-mon[35244]: pgmap v206: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 4.0 MiB/s rd, 790 MiB/s wr, 1.55k op/s 2024-09-10T10:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3315370312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 msgr2=0x7fc4c0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3315370312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0101f30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009930 tx=0x7fc4a802a480 comp rx=0 tx=0).stop 2024-09-10T10:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3315370312 shutdown_connections 2024-09-10T10:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3315370312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4c0102470 0x7fc4c010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3315370312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0101f30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3315370312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4c0107b70 0x7fc4c0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3315370312 >> 172.21.15.45:0/3315370312 conn(0x7fc4c00fd7d0 msgr2=0x7fc4c00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3315370312 shutdown_connections 2024-09-10T10:09:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.635+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3315370312 wait complete. 2024-09-10T10:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.636+0000 7fc4c5e5e640 1 Processor -- start 2024-09-10T10:09:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.637+0000 7fc4c5e5e640 1 -- start start 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.637+0000 7fc4c5e5e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.637+0000 7fc4c5e5e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4c0102470 0x7fc4c0116860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.637+0000 7fc4c5e5e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4c0107b70 0x7fc4c011c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.637+0000 7fc4c5e5e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c010fec0 con 0x7fc4c0101ad0 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.637+0000 7fc4c5e5e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00763a0 con 0x7fc4c0107b70 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.637+0000 7fc4c5e5e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0110040 con 0x7fc4c0102470 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43966/0 (socket says 172.21.15.45:43966) 2024-09-10T10:09:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/3661293587 learned_addr learned my addr 172.21.15.45:0/3661293587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/3661293587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4c0102470 msgr2=0x7fc4c0116860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4c0102470 0x7fc4c0116860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/3661293587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4c0107b70 msgr2=0x7fc4c011c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4c0107b70 0x7fc4c011c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.638+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/3661293587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01114a0 con 0x7fc4c0101ad0 2024-09-10T10:09:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.639+0000 7fc4bf7fe640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0116320 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b450 tx=0x7fc4b000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.639+0000 7fc4bcff9640 1 -- 172.21.15.45:0/3661293587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0015020 con 0x7fc4c0101ad0 2024-09-10T10:09:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.639+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c0111790 con 0x7fc4c0101ad0 2024-09-10T10:09:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.639+0000 7fc4bcff9640 1 -- 172.21.15.45:0/3661293587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b0011d70 con 0x7fc4c0101ad0 2024-09-10T10:09:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.639+0000 7fc4bcff9640 1 -- 172.21.15.45:0/3661293587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000f340 con 0x7fc4c0101ad0 2024-09-10T10:09:58.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.640+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c0111d30 con 0x7fc4c0101ad0 2024-09-10T10:09:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.644+0000 7fc4c8bfe740 1 Processor -- start 2024-09-10T10:09:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.644+0000 7fc4c8bfe740 1 -- start start 2024-09-10T10:09:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.645+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b73ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.645+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.645+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.645+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd24ad82f0 con 0x55dd24b74410 2024-09-10T10:09:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.645+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dd24a80270 con 0x55dd24b73af0 2024-09-10T10:09:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.645+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55dd24ad42b0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.645+0000 7fc4bcff9640 1 -- 172.21.15.45:0/3661293587 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc4b0018020 con 0x7fc4c0101ad0 2024-09-10T10:09:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.646+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.646+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b73ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.646+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43978/0 (socket says 172.21.15.45:43978) 2024-09-10T10:09:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.646+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/4020691274 learned_addr learned my addr 172.21.15.45:0/4020691274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.646+0000 7fc4beffd640 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.647+0000 7fc4beffd640 1 -- 172.21.15.45:0/4020691274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 msgr2=0x55dd24b73ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.647+0000 7fc4beffd640 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b73ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.647+0000 7fc4beffd640 1 -- 172.21.15.45:0/4020691274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 msgr2=0x55dd24b7e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.647+0000 7fc4beffd640 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.647+0000 7fc4beffd640 1 -- 172.21.15.45:0/4020691274 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd24a7f340 con 0x55dd24b7ecb0 2024-09-10T10:09:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.647+0000 7fc4beffd640 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b810a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ffb00 tx=0x7fc4a8005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8006e85b8ae84bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.648+0000 7fc4b5ffb640 1 -- 172.21.15.45:0/4020691274 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8040070 con 0x55dd24b7ecb0 2024-09-10T10:09:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.649+0000 7fc4b5ffb640 1 -- 172.21.15.45:0/4020691274 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a80317b0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.649+0000 7fc4b5ffb640 1 -- 172.21.15.45:0/4020691274 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8035040 con 0x55dd24b7ecb0 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.649+0000 7fc4bcff9640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc4ac0a2e20 0x7fc4ac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4bffff640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc4ac0a2e20 0x7fc4ac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/4020691274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 msgr2=0x55dd24b810a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b810a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ffb00 tx=0x7fc4a8005e40 comp rx=0 tx=0).stop 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/4020691274 shutdown_connections 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b810a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/4020691274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b73ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.652 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/4020691274 >> 172.21.15.45:0/4020691274 conn(0x55dd24b70060 msgr2=0x55dd24b70440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.650+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/4020691274 shutdown_connections 2024-09-10T10:09:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.651+0000 7fc4bcff9640 1 -- 172.21.15.45:0/3661293587 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7fc4b00ceaf0 con 0x7fc4c0101ad0 2024-09-10T10:09:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.651+0000 7fc4bffff640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc4ac0a2e20 0x7fc4ac0a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0116fb0 tx=0x7fc4b800d040 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.651+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/4020691274 wait complete. 2024-09-10T10:09:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.651+0000 7fc4c8bfe740 1 -- start start 2024-09-10T10:09:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.652+0000 7fc4c8bfe740 1 Processor -- start 2024-09-10T10:09:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.652+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.652+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b74410 0x55dd24b7c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.652+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b7c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.652+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd24b81880 con 0x55dd24b7ecb0 2024-09-10T10:09:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.652+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dd24a61290 con 0x55dd24b73af0 2024-09-10T10:09:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.653+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55dd24b81a00 con 0x55dd24b74410 2024-09-10T10:09:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.653+0000 7fc4c8bfe740 10 client.0 _ll_register_callbacks cb 0x55dd24a92140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:09:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.653+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.653+0000 7fc4bf7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:37126/0 (socket says 172.21.15.45:37126) 2024-09-10T10:09:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.653+0000 7fc4bf7fe640 1 -- 172.21.15.45:0/581351016 learned_addr learned my addr 172.21.15.45:0/581351016 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.654+0000 7fc4bffff640 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b7c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.654+0000 7fc4beffd640 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b74410 0x55dd24b7c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.654+0000 7fc4bffff640 1 -- 172.21.15.45:0/581351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b74410 msgr2=0x55dd24b7c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.654+0000 7fc4bffff640 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b74410 0x55dd24b7c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.654+0000 7fc4bffff640 1 -- 172.21.15.45:0/581351016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 msgr2=0x55dd24b7bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.654+0000 7fc4bffff640 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.654+0000 7fc4bffff640 1 -- 172.21.15.45:0/581351016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd247eb210 con 0x55dd24b7ecb0 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.655+0000 7fc4bffff640 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b7c5f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8016e10 tx=0x7fc4b8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ad50b332663de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.655+0000 7fc49effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:09:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.655+0000 7fc49effd640 1 -- 172.21.15.45:0/581351016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8025060 con 0x55dd24b7ecb0 2024-09-10T10:09:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.655+0000 7fc49effd640 1 -- 172.21.15.45:0/581351016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7fc4b80205f0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.655+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a80317b0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.656+0000 7fc49effd640 1 -- 172.21.15.45:0/581351016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8009b70 con 0x55dd24b7ecb0 2024-09-10T10:09:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.656+0000 7fc4c8bfe740 10 client.15531 fetch_fsmap learned FSMap version 21 2024-09-10T10:09:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.656+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd248b7620 con 0x55dd24b7ecb0 2024-09-10T10:09:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.656+0000 7fc49effd640 1 -- 172.21.15.45:0/581351016 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a80317b0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.657+0000 7fc49effd640 1 -- 172.21.15.45:0/581351016 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc4b8028930 con 0x55dd24b7ecb0 2024-09-10T10:09:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.657+0000 7fc4c8bfe740 10 client.15531 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:09:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.657+0000 7fc4c8bfe740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:09:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.657+0000 7fc4c8bfe740 20 client.15531 populate_metadata read hostname 'smithi045' 2024-09-10T10:09:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.657+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24b8f0e0 0x55dd24b91540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.657+0000 7fc4c8bfe740 4 client.15531 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:09:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.657+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd24ae5a20 con 0x55dd24b8f0e0 2024-09-10T10:09:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.658+0000 7fc4bf7fe640 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24b8f0e0 0x55dd24b91540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.660+0000 7fc4bf7fe640 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24b8f0e0 0x55dd24b91540 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d137425fc10bd1d server_cookie=cff0f1fc677d8e46 in_seq=0 out_seq=0 2024-09-10T10:09:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.660+0000 7fc49effd640 10 client.15531 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.665+0000 7fc49effd640 1 -- 172.21.15.45:0/581351016 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd24ae5a20 con 0x55dd24b8f0e0 2024-09-10T10:09:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.665+0000 7fc49effd640 10 client.15531 handle_command_reply: tid=0 2024-09-10T10:09:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.665+0000 7fc4c8bfe740 1 client.15531 shutdown 2024-09-10T10:09:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 msgr2=0x55dd24b7c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b7c5f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8016e10 tx=0x7fc4b8004b40 comp rx=0 tx=0).stop 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 shutdown_connections 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24b8f0e0 0x55dd24b91540 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b7c5f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b74410 0x55dd24b7c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/581351016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 >> 172.21.15.45:0/581351016 conn(0x55dd24b70060 msgr2=0x55dd24b75dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc49effd640 0 client.15531 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 shutdown_connections 2024-09-10T10:09:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/581351016 wait complete. 2024-09-10T10:09:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.666+0000 7fc4c8bfe740 20 client.15531 trim_cache size 0 max 16384 2024-09-10T10:09:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.698+0000 7fc4c8bfe740 1 Processor -- start 2024-09-10T10:09:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4c8bfe740 1 -- start start 2024-09-10T10:09:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b96e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a000c7c0 con 0x55dd24b74410 2024-09-10T10:09:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a000f060 con 0x55dd24b73af0 2024-09-10T10:09:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a000ce50 con 0x55dd24b7ecb0 2024-09-10T10:09:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44006/0 (socket says 172.21.15.45:44006) 2024-09-10T10:09:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.699+0000 7fc4beffd640 1 -- 172.21.15.45:0/2522467386 learned_addr learned my addr 172.21.15.45:0/2522467386 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4bffff640 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b96e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4beffd640 1 -- 172.21.15.45:0/2522467386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 msgr2=0x55dd24b96e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4beffd640 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b96e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4beffd640 1 -- 172.21.15.45:0/2522467386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 msgr2=0x55dd24b7c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4beffd640 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4beffd640 1 -- 172.21.15.45:0/2522467386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd24b6fe90 con 0x55dd24b74410 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4bffff640 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b96e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:09:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4beffd640 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7dae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005940 tx=0x7fc4a8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999d225b50032a10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.700+0000 7fc4b57fa640 1 -- 172.21.15.45:0/2522467386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8040040 con 0x55dd24b74410 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.701+0000 7fc4b57fa640 1 -- 172.21.15.45:0/2522467386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a8009520 con 0x55dd24b74410 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.701+0000 7fc4b57fa640 1 -- 172.21.15.45:0/2522467386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8035040 con 0x55dd24b74410 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/2522467386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 msgr2=0x55dd24b7dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7dae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005940 tx=0x7fc4a8005dc0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/2522467386 shutdown_connections 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b7ecb0 0x55dd24b96e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b74410 0x55dd24b7dae0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/2522467386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b73af0 0x55dd24b7c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.705 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/2522467386 >> 172.21.15.45:0/2522467386 conn(0x55dd24b70060 msgr2=0x55dd24b7d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/2522467386 shutdown_connections 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.702+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/2522467386 wait complete. 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.703+0000 7fc4c8bfe740 1 -- start start 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.703+0000 7fc4c8bfe740 1 Processor -- start 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b73af0 0x55dd24b872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b74410 0x55dd24b78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4c8bfe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b78c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a000d520 con 0x55dd24b7ecb0 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a000e990 con 0x55dd24b74410 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4c8bfe740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a000dbf0 con 0x55dd24b73af0 2024-09-10T10:09:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4c8bfe740 10 client.0 _ll_register_callbacks cb 0x7fc4b800af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:09:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b78c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b78c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44022/0 (socket says 172.21.15.45:44022) 2024-09-10T10:09:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.704+0000 7fc4bffff640 1 -- 172.21.15.45:0/792292680 learned_addr learned my addr 172.21.15.45:0/792292680 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.705+0000 7fc4bffff640 1 -- 172.21.15.45:0/792292680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b73af0 msgr2=0x55dd24b872e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.705+0000 7fc4bffff640 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b73af0 0x55dd24b872e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.705+0000 7fc4bffff640 1 -- 172.21.15.45:0/792292680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b74410 msgr2=0x55dd24b78730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.705+0000 7fc4bffff640 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b74410 0x55dd24b78730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.705+0000 7fc4bffff640 1 -- 172.21.15.45:0/792292680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd24b7e700 con 0x55dd24b7ecb0 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.705+0000 7fc4bffff640 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b78c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8004b10 tx=0x7fc4b80161a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250bee8dd08606b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.705+0000 7fc49effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:09:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.706+0000 7fc49effd640 1 -- 172.21.15.45:0/792292680 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8036020 con 0x55dd24b7ecb0 2024-09-10T10:09:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.706+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a8009520 con 0x55dd24b7ecb0 2024-09-10T10:09:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.706+0000 7fc49effd640 1 -- 172.21.15.45:0/792292680 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8512+0+0 (secure 0 0 0) 0x7fc4b80287b0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.706+0000 7fc49effd640 1 -- 172.21.15.45:0/792292680 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8041050 con 0x55dd24b7ecb0 2024-09-10T10:09:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.707+0000 7fc4c8bfe740 10 client.15537 fetch_fsmap learned FSMap version 21 2024-09-10T10:09:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.707+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55dd24bf9a60 con 0x55dd24b7ecb0 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.707+0000 7fc49effd640 1 -- 172.21.15.45:0/792292680 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b80095d0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.707+0000 7fc49effd640 1 -- 172.21.15.45:0/792292680 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fc4b8027cd0 con 0x55dd24b7ecb0 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.708+0000 7fc4c8bfe740 10 client.15537 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.708+0000 7fc4c8bfe740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.708+0000 7fc4c8bfe740 20 client.15537 populate_metadata read hostname 'smithi045' 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.708+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24bfa1d0 0x55dd24bfc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.708+0000 7fc4c8bfe740 4 client.15537 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.708+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc4a8009520 con 0x55dd24bfa1d0 2024-09-10T10:09:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.709+0000 7fc4bf7fe640 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24bfa1d0 0x55dd24bfc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:09:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.713+0000 7fc49effd640 10 client.15537 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:09:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:09:58.713+0000 7fc4bf7fe640 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24bfa1d0 0x55dd24bfc5f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e7488dd0fbf817 server_cookie=5207be0bac7e2c90 in_seq=0 out_seq=0 2024-09-10T10:09:59.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:09:59 smithi162 ceph-mon[34963]: pgmap v207: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 875 MiB/s wr, 1.78k op/s 2024-09-10T10:09:59.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:09:59 smithi045 ceph-mon[31114]: pgmap v207: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 875 MiB/s wr, 1.78k op/s 2024-09-10T10:09:59.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:09:59 smithi047 ceph-mon[35244]: pgmap v207: 97 pgs: 97 active+clean; 60 GiB data, 325 GiB used, 748 GiB / 1.0 TiB avail; 4.4 MiB/s rd, 875 MiB/s wr, 1.78k op/s 2024-09-10T10:10:00.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:00 smithi045 ceph-mon[31114]: overall HEALTH_OK 2024-09-10T10:10:00.902 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:00 smithi162 ceph-mon[34963]: overall HEALTH_OK 2024-09-10T10:10:00.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:00 smithi047 ceph-mon[35244]: overall HEALTH_OK 2024-09-10T10:10:01.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:01 smithi162 ceph-mon[34963]: pgmap v208: 97 pgs: 97 active+clean; 60 GiB data, 329 GiB used, 744 GiB / 1.0 TiB avail; 4.8 MiB/s rd, 847 MiB/s wr, 1.67k op/s 2024-09-10T10:10:01.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:01 smithi045 ceph-mon[31114]: pgmap v208: 97 pgs: 97 active+clean; 60 GiB data, 329 GiB used, 744 GiB / 1.0 TiB avail; 4.8 MiB/s rd, 847 MiB/s wr, 1.67k op/s 2024-09-10T10:10:01.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:01 smithi047 ceph-mon[35244]: pgmap v208: 97 pgs: 97 active+clean; 60 GiB data, 329 GiB used, 744 GiB / 1.0 TiB avail; 4.8 MiB/s rd, 847 MiB/s wr, 1.67k op/s 2024-09-10T10:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc49effd640 1 -- 172.21.15.45:0/792292680 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc4a8009520 con 0x55dd24bfa1d0 2024-09-10T10:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc49effd640 10 client.15537 handle_command_reply: tid=0 2024-09-10T10:10:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 client.15537 shutdown 2024-09-10T10:10:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 msgr2=0x55dd24b78c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b78c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8004b10 tx=0x7fc4b80161a0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 shutdown_connections 2024-09-10T10:10:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55dd24bfa1d0 0x55dd24bfc5f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd24b7ecb0 0x55dd24b78c70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd24b74410 0x55dd24b78730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 --2- 172.21.15.45:0/792292680 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55dd24b73af0 0x55dd24b872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 >> 172.21.15.45:0/792292680 conn(0x55dd24b70060 msgr2=0x55dd24b6f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc49effd640 0 client.15537 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 shutdown_connections 2024-09-10T10:10:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.981+0000 7fc4c8bfe740 1 -- 172.21.15.45:0/792292680 wait complete. 2024-09-10T10:10:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.982+0000 7fc4c8bfe740 20 client.15537 trim_cache size 0 max 16384 2024-09-10T10:10:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:10:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:10:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:10:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc4ac0a2e20 msgr2=0x7fc4ac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc4ac0a2e20 0x7fc4ac0a52e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0116fb0 tx=0x7fc4b800d040 comp rx=0 tx=0).stop 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 msgr2=0x7fc4c0116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0116320 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000b450 tx=0x7fc4b000b910 comp rx=0 tx=0).stop 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 shutdown_connections 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fc4ac0a2e20 0x7fc4ac0a52e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4c0107b70 0x7fc4c011c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc4c0102470 0x7fc4c0116860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 --2- 172.21.15.45:0/3661293587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c0101ad0 0x7fc4c0116320 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 >> 172.21.15.45:0/3661293587 conn(0x7fc4c00fd7d0 msgr2=0x7fc4c00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 shutdown_connections 2024-09-10T10:10:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:01.983+0000 7fc4c5e5e640 1 -- 172.21.15.45:0/3661293587 wait complete. 2024-09-10T10:10:03.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:03 smithi045 ceph-mon[31114]: pgmap v209: 97 pgs: 97 active+clean; 60 GiB data, 329 GiB used, 744 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 762 MiB/s wr, 1.35k op/s 2024-09-10T10:10:03.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:03 smithi162 ceph-mon[34963]: pgmap v209: 97 pgs: 97 active+clean; 60 GiB data, 329 GiB used, 744 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 762 MiB/s wr, 1.35k op/s 2024-09-10T10:10:03.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:03 smithi047 ceph-mon[35244]: pgmap v209: 97 pgs: 97 active+clean; 60 GiB data, 329 GiB used, 744 GiB / 1.0 TiB avail; 4.1 MiB/s rd, 762 MiB/s wr, 1.35k op/s 2024-09-10T10:10:05.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:05 smithi162 ceph-mon[34963]: pgmap v210: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 6.0 MiB/s rd, 994 MiB/s wr, 2.05k op/s 2024-09-10T10:10:05.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:05 smithi045 ceph-mon[31114]: pgmap v210: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 6.0 MiB/s rd, 994 MiB/s wr, 2.05k op/s 2024-09-10T10:10:05.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:05 smithi047 ceph-mon[35244]: pgmap v210: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 6.0 MiB/s rd, 994 MiB/s wr, 2.05k op/s 2024-09-10T10:10:07.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:07 smithi162 ceph-mon[34963]: pgmap v211: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 771 MiB/s wr, 1.85k op/s 2024-09-10T10:10:07.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:07 smithi045 ceph-mon[31114]: pgmap v211: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 771 MiB/s wr, 1.85k op/s 2024-09-10T10:10:07.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:07 smithi047 ceph-mon[35244]: pgmap v211: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 771 MiB/s wr, 1.85k op/s 2024-09-10T10:10:09.900 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:09 smithi162 ceph-mon[34963]: pgmap v212: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 796 MiB/s wr, 1.99k op/s 2024-09-10T10:10:09.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:09 smithi045 ceph-mon[31114]: pgmap v212: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 796 MiB/s wr, 1.99k op/s 2024-09-10T10:10:09.910 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:09 smithi047 ceph-mon[35244]: pgmap v212: 97 pgs: 97 active+clean; 60 GiB data, 318 GiB used, 754 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 796 MiB/s wr, 1.99k op/s 2024-09-10T10:10:10.899 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:10 smithi162 ceph-mon[34963]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:10:10.900 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:10 smithi045 ceph-mon[31114]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:10:10.909 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:10 smithi047 ceph-mon[35244]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T10:10:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:11 smithi162 ceph-mon[34963]: pgmap v214: 97 pgs: 97 active+clean; 60 GiB data, 321 GiB used, 752 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 688 MiB/s wr, 1.93k op/s 2024-09-10T10:10:12.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:11 smithi045 ceph-mon[31114]: pgmap v214: 97 pgs: 97 active+clean; 60 GiB data, 321 GiB used, 752 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 688 MiB/s wr, 1.93k op/s 2024-09-10T10:10:12.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:11 smithi047 ceph-mon[35244]: pgmap v214: 97 pgs: 97 active+clean; 60 GiB data, 321 GiB used, 752 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 688 MiB/s wr, 1.93k op/s 2024-09-10T10:10:14.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:13 smithi162 ceph-mon[34963]: pgmap v215: 97 pgs: 97 active+clean; 60 GiB data, 321 GiB used, 752 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 607 MiB/s wr, 1.80k op/s 2024-09-10T10:10:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:13 smithi045 ceph-mon[31114]: pgmap v215: 97 pgs: 97 active+clean; 60 GiB data, 321 GiB used, 752 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 607 MiB/s wr, 1.80k op/s 2024-09-10T10:10:14.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:13 smithi047 ceph-mon[35244]: pgmap v215: 97 pgs: 97 active+clean; 60 GiB data, 321 GiB used, 752 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 607 MiB/s wr, 1.80k op/s 2024-09-10T10:10:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 -- 172.21.15.45:0/99616800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f506011b140 msgr2=0x7f506011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 --2- 172.21.15.45:0/99616800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f506011b140 0x7f506011d530 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5058009ca0 tx=0x7f505802a8d0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 -- 172.21.15.45:0/99616800 shutdown_connections 2024-09-10T10:10:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 --2- 172.21.15.45:0/99616800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f506011b140 0x7f506011d530 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 --2- 172.21.15.45:0/99616800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f50601095d0 0x7f5060109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 --2- 172.21.15.45:0/99616800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50601091f0 0x7f5060110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 -- 172.21.15.45:0/99616800 >> 172.21.15.45:0/99616800 conn(0x7f506006ce00 msgr2=0x7f506006d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 -- 172.21.15.45:0/99616800 shutdown_connections 2024-09-10T10:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.198+0000 7f506655b640 1 -- 172.21.15.45:0/99616800 wait complete. 2024-09-10T10:10:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.199+0000 7f506655b640 1 Processor -- start 2024-09-10T10:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.199+0000 7f506655b640 1 -- start start 2024-09-10T10:10:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.200+0000 7f506655b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50601091f0 0x7f5060118580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.200+0000 7f506655b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50601095d0 0x7f5060118ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.200+0000 7f506655b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f506011b140 0x7f50601116b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.200+0000 7f506655b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5060120d30 con 0x7f50601095d0 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.200+0000 7f506655b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5060120bb0 con 0x7f50601091f0 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.200+0000 7f506655b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5060120eb0 con 0x7f506011b140 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.201+0000 7f505ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50601091f0 0x7f5060118580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.202+0000 7f5064af8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f506011b140 0x7f50601116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.202+0000 7f5064af8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f506011b140 0x7f50601116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:51742/0 (socket says 172.21.15.45:51742) 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.202+0000 7f5064af8640 1 -- 172.21.15.45:0/3611130500 learned_addr learned my addr 172.21.15.45:0/3611130500 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.202+0000 7f505f7fe640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50601095d0 0x7f5060118ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.203+0000 7f505f7fe640 1 -- 172.21.15.45:0/3611130500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f506011b140 msgr2=0x7f50601116b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.203+0000 7f505f7fe640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f506011b140 0x7f50601116b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.203+0000 7f505f7fe640 1 -- 172.21.15.45:0/3611130500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50601091f0 msgr2=0x7f5060118580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.203+0000 7f505f7fe640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50601091f0 0x7f5060118580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.203+0000 7f505f7fe640 1 -- 172.21.15.45:0/3611130500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5060111f70 con 0x7f50601095d0 2024-09-10T10:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.203+0000 7f505f7fe640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50601095d0 0x7f5060118ac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f505400b3f0 tx=0x7f505400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.203+0000 7f505d7fa640 1 -- 172.21.15.45:0/3611130500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054015020 con 0x7f50601095d0 2024-09-10T10:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.204+0000 7f505d7fa640 1 -- 172.21.15.45:0/3611130500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5054011d10 con 0x7f50601095d0 2024-09-10T10:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.204+0000 7f505d7fa640 1 -- 172.21.15.45:0/3611130500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505400f2e0 con 0x7f50601095d0 2024-09-10T10:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.204+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5060112260 con 0x7f50601095d0 2024-09-10T10:10:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.204+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5060116280 con 0x7f50601095d0 2024-09-10T10:10:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.208+0000 7f50693cf740 1 Processor -- start 2024-09-10T10:10:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.208+0000 7f50693cf740 1 -- start start 2024-09-10T10:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.208+0000 7f50693cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9aaf0 0x5557daa9aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f50693cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daaa5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f50693cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daaa5cb0 0x5557daaa80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f50693cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557da9ff2f0 con 0x5557daa9aaf0 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f50693cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5557da9a7270 con 0x5557daaa5cb0 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f50693cf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5557da9fb2b0 con 0x5557daa9b410 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9aaf0 0x5557daa9aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f505d7fa640 1 -- 172.21.15.45:0/3611130500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5054018020 con 0x7f50601095d0 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9aaf0 0x5557daa9aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52956/0 (socket says 172.21.15.45:52956) 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 -- 172.21.15.45:0/3887004315 learned_addr learned my addr 172.21.15.45:0/3887004315 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 -- 172.21.15.45:0/3887004315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 msgr2=0x5557daaa5770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 --2- 172.21.15.45:0/3887004315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daaa5770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 -- 172.21.15.45:0/3887004315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daaa5cb0 msgr2=0x5557daaa80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 --2- 172.21.15.45:0/3887004315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daaa5cb0 0x5557daaa80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.209+0000 7f5064af8640 1 -- 172.21.15.45:0/3887004315 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557da9a6340 con 0x5557daa9aaf0 2024-09-10T10:10:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.210+0000 7f5064af8640 1 --2- 172.21.15.45:0/3887004315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9aaf0 0x5557daa9aed0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5058009a40 tx=0x7f5058031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b0d4335fb67c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.210+0000 7f50467fc640 1 -- 172.21.15.45:0/3887004315 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580314e0 con 0x5557daa9aaf0 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.210+0000 7f505d7fa640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f50480a2e70 0x7f50480a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.211+0000 7f505ffff640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f50480a2e70 0x7f50480a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.211+0000 7f505d7fa640 1 -- 172.21.15.45:0/3611130500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7f50540ce850 con 0x7f50601095d0 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.211+0000 7f50467fc640 1 -- 172.21.15.45:0/3887004315 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5058036040 con 0x5557daa9aaf0 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.211+0000 7f50467fc640 1 -- 172.21.15.45:0/3887004315 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058004700 con 0x5557daa9aaf0 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 -- 172.21.15.45:0/3887004315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9aaf0 msgr2=0x5557daa9aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f505ffff640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f50480a2e70 0x7f50480a5330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5050005d90 tx=0x7f5050005d20 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 --2- 172.21.15.45:0/3887004315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9aaf0 0x5557daa9aed0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5058009a40 tx=0x7f5058031b30 comp rx=0 tx=0).stop 2024-09-10T10:10:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 -- 172.21.15.45:0/3887004315 shutdown_connections 2024-09-10T10:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 --2- 172.21.15.45:0/3887004315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daaa5cb0 0x5557daaa80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 --2- 172.21.15.45:0/3887004315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daaa5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 --2- 172.21.15.45:0/3887004315 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9aaf0 0x5557daa9aed0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 -- 172.21.15.45:0/3887004315 >> 172.21.15.45:0/3887004315 conn(0x5557daa97060 msgr2=0x5557daa97440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 -- 172.21.15.45:0/3887004315 shutdown_connections 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.212+0000 7f50693cf740 1 -- 172.21.15.45:0/3887004315 wait complete. 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.213+0000 7f50693cf740 1 -- start start 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.213+0000 7f50693cf740 1 Processor -- start 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f50693cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daaa2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f50693cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daaa30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f50693cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daaa3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f50693cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557daaa8870 con 0x5557daaa5cb0 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f50693cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5557da988290 con 0x5557daa9aaf0 2024-09-10T10:10:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f50693cf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5557daaa89f0 con 0x5557daa9b410 2024-09-10T10:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f50693cf740 10 client.0 _ll_register_callbacks cb 0x5557da9b9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f5064af8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daaa3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f5064af8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daaa3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52972/0 (socket says 172.21.15.45:52972) 2024-09-10T10:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.214+0000 7f5064af8640 1 -- 172.21.15.45:0/1546666327 learned_addr learned my addr 172.21.15.45:0/1546666327 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.215+0000 7f505f7fe640 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daaa30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.215+0000 7f5064af8640 1 -- 172.21.15.45:0/1546666327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 msgr2=0x5557daaa30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.215+0000 7f5064af8640 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daaa30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.215+0000 7f5064af8640 1 -- 172.21.15.45:0/1546666327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 msgr2=0x5557daaa2b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:10:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.215+0000 7f5064af8640 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daaa2b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.215+0000 7f5064af8640 1 -- 172.21.15.45:0/1546666327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557da712210 con 0x5557daaa5cb0 2024-09-10T10:10:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.216+0000 7f5064af8640 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daaa3610 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f505802aec0 tx=0x7f505802a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29aa0b980f4caa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.216+0000 7f503f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.216+0000 7f503f7fe640 1 -- 172.21.15.45:0/1546666327 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058053060 con 0x5557daaa5cb0 2024-09-10T10:10:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.216+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5058036040 con 0x5557daaa5cb0 2024-09-10T10:10:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.217+0000 7f503f7fe640 1 -- 172.21.15.45:0/1546666327 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7f5058041040 con 0x5557daaa5cb0 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.217+0000 7f503f7fe640 1 -- 172.21.15.45:0/1546666327 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5058003b80 con 0x5557daaa5cb0 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.217+0000 7f50693cf740 10 client.15549 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.217+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5557da7de620 con 0x5557daaa5cb0 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f503f7fe640 1 -- 172.21.15.45:0/1546666327 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5058036040 con 0x5557daaa5cb0 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f503f7fe640 1 -- 172.21.15.45:0/1546666327 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f505804d070 con 0x5557daaa5cb0 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f50693cf740 10 client.15549 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f50693cf740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f50693cf740 20 client.15549 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:15.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f50693cf740 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557daab5a70 0x5557daab7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f50693cf740 4 client.15549 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:10:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.218+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557daa0ca20 con 0x5557daab5a70 2024-09-10T10:10:15.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.219+0000 7f505ffff640 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557daab5a70 0x5557daab7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.222+0000 7f505ffff640 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557daab5a70 0x5557daab7e60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf40b6c5667348f server_cookie=9313a2df100793de in_seq=0 out_seq=0 2024-09-10T10:10:15.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.222+0000 7f503f7fe640 10 client.15549 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.223+0000 7f503f7fe640 1 -- 172.21.15.45:0/1546666327 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557daa0ca20 con 0x5557daab5a70 2024-09-10T10:10:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.223+0000 7f503f7fe640 10 client.15549 handle_command_reply: tid=0 2024-09-10T10:10:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 client.15549 shutdown 2024-09-10T10:10:15.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 msgr2=0x5557daaa3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daaa3610 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f505802aec0 tx=0x7f505802a8d0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 shutdown_connections 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557daab5a70 0x5557daab7e60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f503f7fe640 0 client.15549 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daaa3610 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daaa30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 --2- 172.21.15.45:0/1546666327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daaa2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.224+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 >> 172.21.15.45:0/1546666327 conn(0x5557daa97060 msgr2=0x5557daa9cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.225+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 shutdown_connections 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.225+0000 7f50693cf740 1 -- 172.21.15.45:0/1546666327 wait complete. 2024-09-10T10:10:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.225+0000 7f50693cf740 20 client.15549 trim_cache size 0 max 16384 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.244+0000 7f50693cf740 1 Processor -- start 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.244+0000 7f50693cf740 1 -- start start 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f50693cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daaa3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f50693cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daabae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f50693cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daabb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f50693cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b1e70 con 0x5557daaa5cb0 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f50693cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b46d0 con 0x5557daa9aaf0 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f50693cf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b24c0 con 0x5557daa9b410 2024-09-10T10:10:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f5064af8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daabb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f5064af8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daabb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52982/0 (socket says 172.21.15.45:52982) 2024-09-10T10:10:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.245+0000 7f5064af8640 1 -- 172.21.15.45:0/1257219179 learned_addr learned my addr 172.21.15.45:0/1257219179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.246+0000 7f505ffff640 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daaa3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.246+0000 7f5064af8640 1 -- 172.21.15.45:0/1257219179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 msgr2=0x5557daabae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.246+0000 7f5064af8640 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daabae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.246+0000 7f5064af8640 1 -- 172.21.15.45:0/1257219179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 msgr2=0x5557daaa3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.246+0000 7f5064af8640 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daaa3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.246+0000 7f5064af8640 1 -- 172.21.15.45:0/1257219179 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557da86e8a0 con 0x5557daaa5cb0 2024-09-10T10:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.246+0000 7f5064af8640 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daabb3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5058031ad0 tx=0x7f5058009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e43e8343f74bfd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.247+0000 7f5045ffb640 1 -- 172.21.15.45:0/1257219179 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058053040 con 0x5557daaa5cb0 2024-09-10T10:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.247+0000 7f5045ffb640 1 -- 172.21.15.45:0/1257219179 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5058036040 con 0x5557daaa5cb0 2024-09-10T10:10:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.247+0000 7f5045ffb640 1 -- 172.21.15.45:0/1257219179 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058030ab0 con 0x5557daaa5cb0 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.248+0000 7f50693cf740 1 -- 172.21.15.45:0/1257219179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 msgr2=0x5557daabb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.248+0000 7f50693cf740 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daabb3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5058031ad0 tx=0x7f5058009800 comp rx=0 tx=0).stop 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.248+0000 7f50693cf740 1 -- 172.21.15.45:0/1257219179 shutdown_connections 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.248+0000 7f50693cf740 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daaa5cb0 0x5557daabb3c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.248+0000 7f50693cf740 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daa9b410 0x5557daabae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.248+0000 7f50693cf740 1 --2- 172.21.15.45:0/1257219179 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daaa3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.248+0000 7f50693cf740 1 -- 172.21.15.45:0/1257219179 >> 172.21.15.45:0/1257219179 conn(0x5557daa97060 msgr2=0x5557daaa4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:15.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.249+0000 7f50693cf740 1 -- 172.21.15.45:0/1257219179 shutdown_connections 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.249+0000 7f50693cf740 1 -- 172.21.15.45:0/1257219179 wait complete. 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.249+0000 7f50693cf740 1 -- start start 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 1 Processor -- start 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daabb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9b410 0x5557daabb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daaa5cb0 0x5557daab8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b2b90 con 0x5557daa9b410 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b4000 con 0x5557daa9aaf0 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f50601b3260 con 0x5557daaa5cb0 2024-09-10T10:10:15.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.250+0000 7f50693cf740 10 client.0 _ll_register_callbacks cb 0x7f5058009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9b410 0x5557daabb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9b410 0x5557daabb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52984/0 (socket says 172.21.15.45:52984) 2024-09-10T10:10:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 -- 172.21.15.45:0/2710160725 learned_addr learned my addr 172.21.15.45:0/2710160725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f5064af8640 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daaa5cb0 0x5557daab8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505ffff640 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daabb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 -- 172.21.15.45:0/2710160725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daaa5cb0 msgr2=0x5557daab8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daaa5cb0 0x5557daab8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 -- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 msgr2=0x5557daabb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daabb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.251+0000 7f505f7fe640 1 -- 172.21.15.45:0/2710160725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557daab8fc0 con 0x5557daa9b410 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.252+0000 7f505f7fe640 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9b410 0x5557daabb740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f50540cc110 tx=0x7f5054010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1246f3a39a2a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.252+0000 7f503f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.252+0000 7f503f7fe640 1 -- 172.21.15.45:0/2710160725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505409f1e0 con 0x5557daa9b410 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.252+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5058036040 con 0x5557daa9b410 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.253+0000 7f503f7fe640 1 -- 172.21.15.45:0/2710160725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8600+0+0 (secure 0 0 0) 0x7f50540cc1d0 con 0x5557daa9b410 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.253+0000 7f503f7fe640 1 -- 172.21.15.45:0/2710160725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f50540a3aa0 con 0x5557daa9b410 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.253+0000 7f50693cf740 10 client.15555 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.253+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5557dab20050 con 0x5557daa9b410 2024-09-10T10:10:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.254+0000 7f503f7fe640 1 -- 172.21.15.45:0/2710160725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50540974a0 con 0x5557daa9b410 2024-09-10T10:10:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.254+0000 7f503f7fe640 1 -- 172.21.15.45:0/2710160725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5054096b80 con 0x5557daa9b410 2024-09-10T10:10:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.254+0000 7f50693cf740 10 client.15555 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.254+0000 7f50693cf740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.255+0000 7f50693cf740 20 client.15555 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.255+0000 7f50693cf740 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557dab207d0 0x5557dab22bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.255+0000 7f50693cf740 4 client.15555 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:10:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.255+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5058036040 con 0x5557dab207d0 2024-09-10T10:10:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.256+0000 7f505ffff640 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557dab207d0 0x5557dab22bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.261+0000 7f503f7fe640 10 client.15555 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:15.261+0000 7f505ffff640 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557dab207d0 0x5557dab22bf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b8e637b1413803 server_cookie=6c2086d466676e15 in_seq=0 out_seq=0 2024-09-10T10:10:15.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:15 smithi045 ceph-mon[31114]: pgmap v216: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 180 KiB/s rd, 309 MiB/s wr, 998 op/s 2024-09-10T10:10:15.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:15 smithi047 ceph-mon[35244]: pgmap v216: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 180 KiB/s rd, 309 MiB/s wr, 998 op/s 2024-09-10T10:10:15.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:15 smithi162 ceph-mon[34963]: pgmap v216: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 180 KiB/s rd, 309 MiB/s wr, 998 op/s 2024-09-10T10:10:17.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:17 smithi162 ceph-mon[34963]: pgmap v217: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 180 KiB/s rd, 254 MiB/s wr, 914 op/s 2024-09-10T10:10:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:17 smithi162 ceph-mon[34963]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:10:17.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:17 smithi045 ceph-mon[31114]: pgmap v217: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 180 KiB/s rd, 254 MiB/s wr, 914 op/s 2024-09-10T10:10:17.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:17 smithi045 ceph-mon[31114]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:10:17.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:17 smithi047 ceph-mon[35244]: pgmap v217: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 180 KiB/s rd, 254 MiB/s wr, 914 op/s 2024-09-10T10:10:17.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:17 smithi047 ceph-mon[35244]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.433+0000 7f503f7fe640 1 -- 172.21.15.45:0/2710160725 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5058036040 con 0x5557dab207d0 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.433+0000 7f503f7fe640 10 client.15555 handle_command_reply: tid=0 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.433+0000 7f50693cf740 1 client.15555 shutdown 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9b410 msgr2=0x5557daabb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9b410 0x5557daabb740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f50540cc110 tx=0x7f5054010eb0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 shutdown_connections 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5557dab207d0 0x5557dab22bf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5557daaa5cb0 0x5557daab8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f503f7fe640 0 client.15555 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557daa9b410 0x5557daabb740 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 --2- 172.21.15.45:0/2710160725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557daa9aaf0 0x5557daabb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 >> 172.21.15.45:0/2710160725 conn(0x5557daa97060 msgr2=0x5557daaa1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 shutdown_connections 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 1 -- 172.21.15.45:0/2710160725 wait complete. 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.434+0000 7f50693cf740 20 client.15555 trim_cache size 0 max 16384 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:10:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:10:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:10:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:10:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f50480a2e70 msgr2=0x7f50480a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f50480a2e70 0x7f50480a5330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5050005d90 tx=0x7f5050005d20 comp rx=0 tx=0).stop 2024-09-10T10:10:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50601095d0 msgr2=0x7f5060118ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50601095d0 0x7f5060118ac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f505400b3f0 tx=0x7f505400b8b0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 shutdown_connections 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f506011b140 0x7f50601116b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50601095d0 0x7f5060118ac0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f50480a2e70 0x7f50480a5330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.435+0000 7f506655b640 1 --2- 172.21.15.45:0/3611130500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50601091f0 0x7f5060118580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.436+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 >> 172.21.15.45:0/3611130500 conn(0x7f506006ce00 msgr2=0x7f506011b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.436+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 shutdown_connections 2024-09-10T10:10:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:19.436+0000 7f506655b640 1 -- 172.21.15.45:0/3611130500 wait complete. 2024-09-10T10:10:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:19 smithi162 ceph-mon[34963]: pgmap v219: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 123 KiB/s rd, 139 MiB/s wr, 926 op/s 2024-09-10T10:10:19.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:19 smithi045 ceph-mon[31114]: pgmap v219: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 123 KiB/s rd, 139 MiB/s wr, 926 op/s 2024-09-10T10:10:19.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:19 smithi047 ceph-mon[35244]: pgmap v219: 97 pgs: 97 active+clean; 60 GiB data, 300 GiB used, 773 GiB / 1.0 TiB avail; 123 KiB/s rd, 139 MiB/s wr, 926 op/s 2024-09-10T10:10:21.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:21 smithi162 ceph-mon[34963]: pgmap v220: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 110 KiB/s rd, 157 MiB/s wr, 1.07k op/s 2024-09-10T10:10:21.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:21 smithi045 ceph-mon[31114]: pgmap v220: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 110 KiB/s rd, 157 MiB/s wr, 1.07k op/s 2024-09-10T10:10:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:21 smithi047 ceph-mon[35244]: pgmap v220: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 110 KiB/s rd, 157 MiB/s wr, 1.07k op/s 2024-09-10T10:10:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:23 smithi162 ceph-mon[34963]: pgmap v221: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 110 KiB/s rd, 149 MiB/s wr, 1.05k op/s 2024-09-10T10:10:23.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:23 smithi045 ceph-mon[31114]: pgmap v221: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 110 KiB/s rd, 149 MiB/s wr, 1.05k op/s 2024-09-10T10:10:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:23 smithi047 ceph-mon[35244]: pgmap v221: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 110 KiB/s rd, 149 MiB/s wr, 1.05k op/s 2024-09-10T10:10:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:25 smithi162 ceph-mon[34963]: pgmap v222: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 23 KiB/s rd, 116 MiB/s wr, 854 op/s 2024-09-10T10:10:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:25 smithi045 ceph-mon[31114]: pgmap v222: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 23 KiB/s rd, 116 MiB/s wr, 854 op/s 2024-09-10T10:10:25.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:25 smithi047 ceph-mon[35244]: pgmap v222: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 23 KiB/s rd, 116 MiB/s wr, 854 op/s 2024-09-10T10:10:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:27 smithi162 ceph-mon[34963]: pgmap v223: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 23 KiB/s rd, 86 MiB/s wr, 645 op/s 2024-09-10T10:10:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:10:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:10:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:10:27.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:10:27.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:27 smithi045 ceph-mon[31114]: pgmap v223: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 23 KiB/s rd, 86 MiB/s wr, 645 op/s 2024-09-10T10:10:27.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:10:27.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:10:27.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:10:27.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:10:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:27 smithi047 ceph-mon[35244]: pgmap v223: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 23 KiB/s rd, 86 MiB/s wr, 645 op/s 2024-09-10T10:10:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:10:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:10:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:10:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:10:29.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:29 smithi162 ceph-mon[34963]: pgmap v224: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 20 KiB/s rd, 73 MiB/s wr, 547 op/s 2024-09-10T10:10:29.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:29 smithi045 ceph-mon[31114]: pgmap v224: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 20 KiB/s rd, 73 MiB/s wr, 547 op/s 2024-09-10T10:10:29.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:29 smithi047 ceph-mon[35244]: pgmap v224: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 20 KiB/s rd, 73 MiB/s wr, 547 op/s 2024-09-10T10:10:31.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:31 smithi162 ceph-mon[34963]: pgmap v225: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 251 op/s 2024-09-10T10:10:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:31 smithi045 ceph-mon[31114]: pgmap v225: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 251 op/s 2024-09-10T10:10:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:31 smithi047 ceph-mon[35244]: pgmap v225: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 21 KiB/s rd, 33 MiB/s wr, 251 op/s 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 -- 172.21.15.45:0/2686954391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c102ca0 msgr2=0x7fb00c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 --2- 172.21.15.45:0/2686954391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c102ca0 0x7fb00c103080 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faffc009900 tx=0x7faffc02a410 comp rx=0 tx=0).stop 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 -- 172.21.15.45:0/2686954391 shutdown_connections 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 --2- 172.21.15.45:0/2686954391 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c10c0d0 0x7fb00c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 --2- 172.21.15.45:0/2686954391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1035c0 0x7fb00c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 --2- 172.21.15.45:0/2686954391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c102ca0 0x7fb00c103080 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 -- 172.21.15.45:0/2686954391 >> 172.21.15.45:0/2686954391 conn(0x7fb00c0fc6b0 msgr2=0x7fb00c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 -- 172.21.15.45:0/2686954391 shutdown_connections 2024-09-10T10:10:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.579+0000 7fb0115af640 1 -- 172.21.15.45:0/2686954391 wait complete. 2024-09-10T10:10:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 Processor -- start 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 -- start start 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c102ca0 0x7fb00c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c1035c0 0x7fb00c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c10c0d0 0x7fb00c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c111f60 con 0x7fb00c10c0d0 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c111de0 con 0x7fb00c102ca0 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb0115af640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1120e0 con 0x7fb00c1035c0 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c10c0d0 0x7fb00c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c10c0d0 0x7fb00c101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46114/0 (socket says 172.21.15.45:46114) 2024-09-10T10:10:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb00b7fe640 1 -- 172.21.15.45:0/1672591410 learned_addr learned my addr 172.21.15.45:0/1672591410 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c1035c0 0x7fb00c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.580+0000 7fb00affd640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c102ca0 0x7fb00c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb00b7fe640 1 -- 172.21.15.45:0/1672591410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c1035c0 msgr2=0x7fb00c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c1035c0 0x7fb00c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb00b7fe640 1 -- 172.21.15.45:0/1672591410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c102ca0 msgr2=0x7fb00c1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c102ca0 0x7fb00c1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb00b7fe640 1 -- 172.21.15.45:0/1672591410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c1a6870 con 0x7fb00c10c0d0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c10c0d0 0x7fb00c101370 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b970 tx=0x7fb00400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7faff3fff640 1 -- 172.21.15.45:0/1672591410 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004019070 con 0x7fb00c10c0d0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1a6b60 con 0x7fb00c10c0d0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7faff3fff640 1 -- 172.21.15.45:0/1672591410 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb00400cb40 con 0x7fb00c10c0d0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7faff3fff640 1 -- 172.21.15.45:0/1672591410 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb004007650 con 0x7fb00c10c0d0 2024-09-10T10:10:32.585 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.581+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a70b0 con 0x7fb00c10c0d0 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.584+0000 7fb014424740 1 Processor -- start 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.584+0000 7faff3fff640 1 -- 172.21.15.45:0/1672591410 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb00401c070 con 0x7fb00c10c0d0 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.584+0000 7fb014424740 1 -- start start 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.584+0000 7fb014424740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d242ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.584+0000 7fb014424740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d243410 0x55a92d24d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb014424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d2500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb014424740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a92d1a72f0 con 0x55a92d24dcb0 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb014424740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a92d14f270 con 0x55a92d243410 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb014424740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a92d1a32b0 con 0x55a92d242af0 2024-09-10T10:10:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d242ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d242ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:44114/0 (socket says 172.21.15.45:44114) 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00a7fc640 1 -- 172.21.15.45:0/575050073 learned_addr learned my addr 172.21.15.45:0/575050073 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00affd640 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d243410 0x55a92d24d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d2500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00affd640 1 -- 172.21.15.45:0/575050073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 msgr2=0x55a92d242ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00affd640 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d242ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00affd640 1 -- 172.21.15.45:0/575050073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 msgr2=0x55a92d2500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00affd640 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d2500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00affd640 1 -- 172.21.15.45:0/575050073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92d14e340 con 0x55a92d243410 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7faff3fff640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faff40a2da0 0x7faff40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00affd640 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d243410 0x55a92d24d770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faffc02aa00 tx=0x7faffc0040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d7e1ffeb0d52d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7faff3fff640 1 -- 172.21.15.45:0/1672591410 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fb0040ce940 con 0x7fb00c10c0d0 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.585+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faff40a2da0 0x7faff40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.586+0000 7faff17fa640 1 -- 172.21.15.45:0/575050073 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc035040 con 0x55a92d243410 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.586+0000 7faff17fa640 1 -- 172.21.15.45:0/575050073 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc005520 con 0x55a92d243410 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.586+0000 7faff17fa640 1 -- 172.21.15.45:0/575050073 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc003850 con 0x55a92d243410 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.586+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faff40a2da0 0x7faff40a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55a92d2403e0 tx=0x7fb0000046c0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.586+0000 7fb014424740 1 -- 172.21.15.45:0/575050073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d243410 msgr2=0x55a92d24d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.586+0000 7fb014424740 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d243410 0x55a92d24d770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faffc02aa00 tx=0x7faffc0040f0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- 172.21.15.45:0/575050073 shutdown_connections 2024-09-10T10:10:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d2500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d243410 0x55a92d24d770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 --2- 172.21.15.45:0/575050073 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d242ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- 172.21.15.45:0/575050073 >> 172.21.15.45:0/575050073 conn(0x55a92d23f060 msgr2=0x55a92d23f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- 172.21.15.45:0/575050073 shutdown_connections 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- 172.21.15.45:0/575050073 wait complete. 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- start start 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 Processor -- start 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d24ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d24b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d24b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a92d250880 con 0x55a92d24dcb0 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a92d130290 con 0x55a92d242af0 2024-09-10T10:10:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55a92d250a00 con 0x55a92d243410 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb014424740 10 client.0 _ll_register_callbacks cb 0x55a92d161140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d24b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d24b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46138/0 (socket says 172.21.15.45:46138) 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d24b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00b7fe640 1 -- 172.21.15.45:0/282156072 learned_addr learned my addr 172.21.15.45:0/282156072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.587+0000 7fb00affd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d24ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00b7fe640 1 -- 172.21.15.45:0/282156072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 msgr2=0x55a92d24b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d24b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00b7fe640 1 -- 172.21.15.45:0/282156072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 msgr2=0x55a92d24ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d24ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00b7fe640 1 -- 172.21.15.45:0/282156072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92ceba210 con 0x55a92d24dcb0 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d24b5c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb0040979d0 tx=0x7fb00409aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7377d9e035249b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fafeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fafeeff5640 1 -- 172.21.15.45:0/282156072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040a3bc0 con 0x55a92d24dcb0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fafeeff5640 1 -- 172.21.15.45:0/282156072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fb0040a3730 con 0x55a92d24dcb0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faffc005520 con 0x55a92d24dcb0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fafeeff5640 1 -- 172.21.15.45:0/282156072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb0040a3930 con 0x55a92d24dcb0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb014424740 10 client.15567 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.588+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a92cf86620 con 0x55a92d24dcb0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fafeeff5640 1 -- 172.21.15.45:0/282156072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faffc005520 con 0x55a92d24dcb0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fafeeff5640 1 -- 172.21.15.45:0/282156072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb004095c90 con 0x55a92d24dcb0 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fb014424740 10 client.15567 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fb014424740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fb014424740 20 client.15567 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fb014424740 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fb014424740 4 client.15567 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:10:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.589+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a92d1b4a20 con 0x55a92d25dc50 2024-09-10T10:10:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.590+0000 7fb00affd640 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.590+0000 7fafeeff5640 10 client.15567 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.590+0000 7fb00affd640 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2600a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dbb994437a9d275 server_cookie=978f4ddfa44889c2 in_seq=0 out_seq=0 2024-09-10T10:10:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fafeeff5640 1 -- 172.21.15.45:0/282156072 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a92d1b4a20 con 0x55a92d25dc50 2024-09-10T10:10:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fafeeff5640 10 client.15567 handle_command_reply: tid=0 2024-09-10T10:10:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 client.15567 shutdown 2024-09-10T10:10:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 msgr2=0x55a92d24b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d24b5c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb0040979d0 tx=0x7fb00409aa50 comp rx=0 tx=0).stop 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 shutdown_connections 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2600a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d24b5c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fafeeff5640 0 client.15567 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d24b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 --2- 172.21.15.45:0/282156072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d24ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 >> 172.21.15.45:0/282156072 conn(0x55a92d23f060 msgr2=0x55a92d244d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 shutdown_connections 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.591+0000 7fb014424740 1 -- 172.21.15.45:0/282156072 wait complete. 2024-09-10T10:10:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.592+0000 7fb014424740 20 client.15567 trim_cache size 0 max 16384 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.611+0000 7fb014424740 1 Processor -- start 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.611+0000 7fb014424740 1 -- start start 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.611+0000 7fb014424740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d24b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.611+0000 7fb014424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d243410 0x55a92d244c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.611+0000 7fb014424740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d24dcb0 0x55a92d2659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb014424740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1ad9c0 con 0x55a92d243410 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb014424740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1b0220 con 0x55a92d24dcb0 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb014424740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1ae010 con 0x55a92d242af0 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d243410 0x55a92d244c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d24b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d243410 0x55a92d244c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46152/0 (socket says 172.21.15.45:46152) 2024-09-10T10:10:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1916408168 learned_addr learned my addr 172.21.15.45:0/1916408168 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1916408168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 msgr2=0x55a92d24b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/1916408168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d24b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1916408168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d24dcb0 msgr2=0x55a92d2659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/1916408168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d24dcb0 0x55a92d2659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 -- 172.21.15.45:0/1916408168 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a92d23ee90 con 0x55a92d243410 2024-09-10T10:10:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.612+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/1916408168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d243410 0x55a92d244c60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55a92d245690 tx=0x7fb00001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdad24dfedee1880 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.613+0000 7faff0ff9640 1 -- 172.21.15.45:0/1916408168 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000003c70 con 0x55a92d243410 2024-09-10T10:10:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.613+0000 7faff0ff9640 1 -- 172.21.15.45:0/1916408168 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb000003e10 con 0x55a92d243410 2024-09-10T10:10:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.613+0000 7faff0ff9640 1 -- 172.21.15.45:0/1916408168 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb000020670 con 0x55a92d243410 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 -- 172.21.15.45:0/1916408168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d243410 msgr2=0x55a92d244c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 --2- 172.21.15.45:0/1916408168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d243410 0x55a92d244c60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55a92d245690 tx=0x7fb00001e000 comp rx=0 tx=0).stop 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 -- 172.21.15.45:0/1916408168 shutdown_connections 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 --2- 172.21.15.45:0/1916408168 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d24dcb0 0x55a92d2659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 --2- 172.21.15.45:0/1916408168 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d243410 0x55a92d244c60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 --2- 172.21.15.45:0/1916408168 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d242af0 0x55a92d24b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 -- 172.21.15.45:0/1916408168 >> 172.21.15.45:0/1916408168 conn(0x55a92d23f060 msgr2=0x55a92d244550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 -- 172.21.15.45:0/1916408168 shutdown_connections 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 -- 172.21.15.45:0/1916408168 wait complete. 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 -- start start 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.614+0000 7fb014424740 1 Processor -- start 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb014424740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d255c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb014424740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d2561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb014424740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d252c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb014424740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1ae6e0 con 0x55a92d24dcb0 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb014424740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1afb50 con 0x55a92d242af0 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb014424740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1aedb0 con 0x55a92d243410 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb014424740 10 client.0 _ll_register_callbacks cb 0x7fb0000095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d252c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d252c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46160/0 (socket says 172.21.15.45:46160) 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 -- 172.21.15.45:0/2087391484 learned_addr learned my addr 172.21.15.45:0/2087391484 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00a7fc640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d2561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00affd640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d255c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 -- 172.21.15.45:0/2087391484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 msgr2=0x55a92d2561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d2561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 -- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 msgr2=0x55a92d255c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d255c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00b7fe640 1 -- 172.21.15.45:0/2087391484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92d2531a0 con 0x55a92d24dcb0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.615+0000 7fb00affd640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d255c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.616+0000 7fb00b7fe640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d252c60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb00409ad00 tx=0x7fb0040972a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e0e3879fbb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.616+0000 7fafeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.616+0000 7fafeeff5640 1 -- 172.21.15.45:0/2087391484 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040a3bc0 con 0x55a92d24dcb0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.616+0000 7fafeeff5640 1 -- 172.21.15.45:0/2087391484 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7fb004097670 con 0x55a92d24dcb0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.616+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb000003e10 con 0x55a92d24dcb0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.616+0000 7fafeeff5640 1 -- 172.21.15.45:0/2087391484 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb004009040 con 0x55a92d24dcb0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.616+0000 7fb014424740 10 client.15573 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55a92d2c9100 con 0x55a92d24dcb0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fafeeff5640 1 -- 172.21.15.45:0/2087391484 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb000003e10 con 0x55a92d24dcb0 2024-09-10T10:10:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fafeeff5640 1 -- 172.21.15.45:0/2087391484 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb004095b70 con 0x55a92d24dcb0 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb014424740 10 client.15573 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb014424740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb014424740 20 client.15573 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb014424740 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb014424740 4 client.15573 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a92d1aed70 con 0x55a92d25dc50 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.617+0000 7fb00affd640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.618+0000 7fafeeff5640 10 client.15573 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.618+0000 7fb00affd640 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2cb7a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c3b1f09ca2331d server_cookie=70f355caab3f2823 in_seq=0 out_seq=0 2024-09-10T10:10:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fafeeff5640 1 -- 172.21.15.45:0/2087391484 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a92d1aed70 con 0x55a92d25dc50 2024-09-10T10:10:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fafeeff5640 10 client.15573 handle_command_reply: tid=0 2024-09-10T10:10:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 client.15573 shutdown 2024-09-10T10:10:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 msgr2=0x55a92d252c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d252c60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb00409ad00 tx=0x7fb0040972a0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 shutdown_connections 2024-09-10T10:10:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55a92d25dc50 0x55a92d2cb7a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a92d24dcb0 0x55a92d252c60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55a92d243410 0x55a92d2561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fafeeff5640 0 client.15573 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 --2- 172.21.15.45:0/2087391484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a92d242af0 0x55a92d255c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 >> 172.21.15.45:0/2087391484 conn(0x55a92d23f060 msgr2=0x55a92d23e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 shutdown_connections 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 1 -- 172.21.15.45:0/2087391484 wait complete. 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.659+0000 7fb014424740 20 client.15573 trim_cache size 0 max 16384 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:10:32.662 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:10:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:10:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:10:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faff40a2da0 msgr2=0x7faff40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.663 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faff40a2da0 0x7faff40a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55a92d2403e0 tx=0x7fb0000046c0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c10c0d0 msgr2=0x7fb00c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c10c0d0 0x7fb00c101370 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b970 tx=0x7fb00400be30 comp rx=0 tx=0).stop 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 shutdown_connections 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb00c10c0d0 0x7fb00c101370 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb00c1035c0 0x7fb00c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faff40a2da0 0x7faff40a5260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 --2- 172.21.15.45:0/1672591410 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c102ca0 0x7fb00c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 >> 172.21.15.45:0/1672591410 conn(0x7fb00c0fc6b0 msgr2=0x7fb00c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 shutdown_connections 2024-09-10T10:10:32.664 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:32.660+0000 7fb0115af640 1 -- 172.21.15.45:0/1672591410 wait complete. 2024-09-10T10:10:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:33 smithi162 ceph-mon[34963]: pgmap v226: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 15 KiB/s rd, 125 KiB/s wr, 7 op/s 2024-09-10T10:10:33.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:33 smithi045 ceph-mon[31114]: pgmap v226: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 15 KiB/s rd, 125 KiB/s wr, 7 op/s 2024-09-10T10:10:33.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:33 smithi047 ceph-mon[35244]: pgmap v226: 97 pgs: 97 active+clean; 60 GiB data, 301 GiB used, 772 GiB / 1.0 TiB avail; 15 KiB/s rd, 125 KiB/s wr, 7 op/s 2024-09-10T10:10:35.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:35 smithi162 ceph-mon[34963]: pgmap v227: 97 pgs: 97 active+clean; 60 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 202 op/s 2024-09-10T10:10:35.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:35 smithi045 ceph-mon[31114]: pgmap v227: 97 pgs: 97 active+clean; 60 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 202 op/s 2024-09-10T10:10:35.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:35 smithi047 ceph-mon[35244]: pgmap v227: 97 pgs: 97 active+clean; 60 GiB data, 226 GiB used, 847 GiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 202 op/s 2024-09-10T10:10:37.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:37 smithi162 ceph-mon[34963]: pgmap v228: 97 pgs: 97 active+clean; 60 GiB data, 226 GiB used, 846 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 28 MiB/s wr, 258 op/s 2024-09-10T10:10:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:37 smithi045 ceph-mon[31114]: pgmap v228: 97 pgs: 97 active+clean; 60 GiB data, 226 GiB used, 846 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 28 MiB/s wr, 258 op/s 2024-09-10T10:10:37.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:37 smithi047 ceph-mon[35244]: pgmap v228: 97 pgs: 97 active+clean; 60 GiB data, 226 GiB used, 846 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 28 MiB/s wr, 258 op/s 2024-09-10T10:10:39.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:39 smithi162 ceph-mon[34963]: pgmap v229: 97 pgs: 97 active+clean; 60 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:39.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:39 smithi045 ceph-mon[31114]: pgmap v229: 97 pgs: 97 active+clean; 60 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:39.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:39 smithi047 ceph-mon[35244]: pgmap v229: 97 pgs: 97 active+clean; 60 GiB data, 227 GiB used, 846 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:41.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:41 smithi162 ceph-mon[34963]: pgmap v230: 97 pgs: 97 active+clean; 60 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:41.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:41 smithi045 ceph-mon[31114]: pgmap v230: 97 pgs: 97 active+clean; 60 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:41.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:41 smithi047 ceph-mon[35244]: pgmap v230: 97 pgs: 97 active+clean; 60 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:43.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:43 smithi162 ceph-mon[34963]: pgmap v231: 97 pgs: 97 active+clean; 60 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 MiB/s wr, 295 op/s 2024-09-10T10:10:43.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:43 smithi045 ceph-mon[31114]: pgmap v231: 97 pgs: 97 active+clean; 60 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 MiB/s wr, 295 op/s 2024-09-10T10:10:43.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:43 smithi047 ceph-mon[35244]: pgmap v231: 97 pgs: 97 active+clean; 60 GiB data, 197 GiB used, 876 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 32 MiB/s wr, 295 op/s 2024-09-10T10:10:45.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:45 smithi162 ceph-mon[34963]: pgmap v232: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:45.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:45 smithi045 ceph-mon[31114]: pgmap v232: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:45.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:45 smithi047 ceph-mon[35244]: pgmap v232: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 5.9 KiB/s rd, 32 MiB/s wr, 296 op/s 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 -- 172.21.15.45:0/2186369461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4102470 msgr2=0x7f81a410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 --2- 172.21.15.45:0/2186369461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4102470 0x7f81a410e9a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f819c009cd0 tx=0x7f819c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 -- 172.21.15.45:0/2186369461 shutdown_connections 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 --2- 172.21.15.45:0/2186369461 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4102470 0x7f81a410e9a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 --2- 172.21.15.45:0/2186369461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81a4101ad0 0x7f81a4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 --2- 172.21.15.45:0/2186369461 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a4107b70 0x7f81a4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 -- 172.21.15.45:0/2186369461 >> 172.21.15.45:0/2186369461 conn(0x7f81a40fd7d0 msgr2=0x7f81a40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 -- 172.21.15.45:0/2186369461 shutdown_connections 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.759+0000 7f81aa853640 1 -- 172.21.15.45:0/2186369461 wait complete. 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 Processor -- start 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 -- start start 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4101ad0 0x7f81a4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a4102470 0x7f81a4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81a4107b70 0x7f81a419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4111f20 con 0x7f81a4101ad0 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4111da0 con 0x7f81a4107b70 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81aa853640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41120a0 con 0x7f81a4102470 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4101ad0 0x7f81a4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4101ad0 0x7f81a4198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58296/0 (socket says 172.21.15.45:58296) 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81a3fff640 1 -- 172.21.15.45:0/1036492017 learned_addr learned my addr 172.21.15.45:0/1036492017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f81a8df0640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81a4107b70 0x7f81a419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.760+0000 7f819bfff640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a4102470 0x7f81a4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a3fff640 1 -- 172.21.15.45:0/1036492017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a4102470 msgr2=0x7f81a4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a3fff640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a4102470 0x7f81a4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a3fff640 1 -- 172.21.15.45:0/1036492017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81a4107b70 msgr2=0x7f81a419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a3fff640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81a4107b70 0x7f81a419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a3fff640 1 -- 172.21.15.45:0/1036492017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a419d220 con 0x7f81a4101ad0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a3fff640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4101ad0 0x7f81a4198230 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f819400c920 tx=0x7f819400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a1ffb640 1 -- 172.21.15.45:0/1036492017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194007c70 con 0x7f81a4101ad0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a1ffb640 1 -- 172.21.15.45:0/1036492017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8194007e10 con 0x7f81a4101ad0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a419d510 con 0x7f81a4101ad0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81a1ffb640 1 -- 172.21.15.45:0/1036492017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194018890 con 0x7f81a4101ad0 2024-09-10T10:10:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.761+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41a4da0 con 0x7f81a4101ad0 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.763+0000 7f81a1ffb640 1 -- 172.21.15.45:0/1036492017 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8194018a70 con 0x7f81a4101ad0 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 Processor -- start 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 -- start start 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd529f0 0x56021cd52dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd61b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56021ccb72f0 con 0x56021cd53310 2024-09-10T10:10:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56021cc5f270 con 0x56021cd620c0 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56021ccb32b0 con 0x56021cd529f0 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd61b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd61b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58308/0 (socket says 172.21.15.45:58308) 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81a3fff640 1 -- 172.21.15.45:0/1434828683 learned_addr learned my addr 172.21.15.45:0/1434828683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f819bfff640 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81a8df0640 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd529f0 0x56021cd52dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.764+0000 7f81a1ffb640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f81900a2e20 0x7f81900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a3fff640 1 -- 172.21.15.45:0/1434828683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd529f0 msgr2=0x56021cd52dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a8df0640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f81900a2e20 0x7f81900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a3fff640 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd529f0 0x56021cd52dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a3fff640 1 -- 172.21.15.45:0/1434828683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 msgr2=0x56021cd644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a3fff640 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a3fff640 1 -- 172.21.15.45:0/1434828683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021cc5e340 con 0x56021cd53310 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f819bfff640 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a1ffb640 1 -- 172.21.15.45:0/1036492017 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f81940cf450 con 0x7f81a4101ad0 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a3fff640 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd61b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f819409b5f0 tx=0x7f8194000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b9ced6e6919e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f819a7fc640 1 -- 172.21.15.45:0/1434828683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819409b250 con 0x56021cd53310 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f81a8df0640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f81900a2e20 0x7f81900a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f819c004ab0 tx=0x7f819c02a8e0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f819a7fc640 1 -- 172.21.15.45:0/1434828683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819409bff0 con 0x56021cd53310 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.765+0000 7f819a7fc640 1 -- 172.21.15.45:0/1434828683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194012110 con 0x56021cd53310 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 -- 172.21.15.45:0/1434828683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 msgr2=0x56021cd61b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd61b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f819409b5f0 tx=0x7f8194000bc0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 -- 172.21.15.45:0/1434828683 shutdown_connections 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd61b80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/1434828683 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd529f0 0x56021cd52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 -- 172.21.15.45:0/1434828683 >> 172.21.15.45:0/1434828683 conn(0x56021cd4f060 msgr2=0x56021cd4f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 -- 172.21.15.45:0/1434828683 shutdown_connections 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 -- 172.21.15.45:0/1434828683 wait complete. 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.766+0000 7f81ad5ec740 1 -- start start 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 1 Processor -- start 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd529f0 0x56021cd5f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd5f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cd5fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56021cd64d20 con 0x56021cd53310 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56021cc40290 con 0x56021cd529f0 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56021cd64ea0 con 0x56021cd620c0 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81ad5ec740 10 client.0 _ll_register_callbacks cb 0x56021cc71140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd5f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd5f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58310/0 (socket says 172.21.15.45:58310) 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 -- 172.21.15.45:0/3344243800 learned_addr learned my addr 172.21.15.45:0/3344243800 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81a3fff640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd529f0 0x56021cd5f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81a8df0640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cd5fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 -- 172.21.15.45:0/3344243800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 msgr2=0x56021cd5fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cd5fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 -- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd529f0 msgr2=0x56021cd5f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd529f0 0x56021cd5f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f819bfff640 1 -- 172.21.15.45:0/3344243800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021c9ca210 con 0x56021cd53310 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81a3fff640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd529f0 0x56021cd5f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.767+0000 7f81a8df0640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cd5fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f819bfff640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd5f540 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f818c012700 tx=0x7f818c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283e5090853de7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81837fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81837fe640 1 -- 172.21.15.45:0/3344243800 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c004030 con 0x56021cd53310 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81837fe640 1 -- 172.21.15.45:0/3344243800 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f818c0041d0 con 0x56021cd53310 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819409bff0 con 0x56021cd53310 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81837fe640 1 -- 172.21.15.45:0/3344243800 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f818c0043d0 con 0x56021cd53310 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 10 client.15585 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56021ca96620 con 0x56021cd53310 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81837fe640 1 -- 172.21.15.45:0/3344243800 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819409bff0 con 0x56021cd53310 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81837fe640 1 -- 172.21.15.45:0/3344243800 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f818c003cb0 con 0x56021cd53310 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 10 client.15585 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 20 client.15585 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cd72130 0x56021cd74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 4 client.15585 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.768+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56021ccc4a20 con 0x56021cd72130 2024-09-10T10:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.769+0000 7f81a3fff640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cd72130 0x56021cd74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.769+0000 7f81a3fff640 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cd72130 0x56021cd74560 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28823cef5ad496ef server_cookie=46cd0b88642d3c88 in_seq=0 out_seq=0 2024-09-10T10:10:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.769+0000 7f81837fe640 10 client.15585 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81837fe640 1 -- 172.21.15.45:0/3344243800 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56021ccc4a20 con 0x56021cd72130 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81837fe640 10 client.15585 handle_command_reply: tid=0 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 client.15585 shutdown 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 msgr2=0x56021cd5f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd5f540 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f818c012700 tx=0x7f818c012bc0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 shutdown_connections 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cd72130 0x56021cd74560 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cd5fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81837fe640 0 client.15585 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd53310 0x56021cd5f540 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/3344243800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd529f0 0x56021cd5f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 >> 172.21.15.45:0/3344243800 conn(0x56021cd4f060 msgr2=0x56021cd54dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 shutdown_connections 2024-09-10T10:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 1 -- 172.21.15.45:0/3344243800 wait complete. 2024-09-10T10:10:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.770+0000 7f81ad5ec740 20 client.15585 trim_cache size 0 max 16384 2024-09-10T10:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 Processor -- start 2024-09-10T10:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 -- start start 2024-09-10T10:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd60f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd53310 0x56021cd79e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd7a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41a95d0 con 0x56021cd529f0 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41abe10 con 0x56021cd620c0 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41a9c20 con 0x56021cd53310 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd60f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd60f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58316/0 (socket says 172.21.15.45:58316) 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 -- 172.21.15.45:0/4210084248 learned_addr learned my addr 172.21.15.45:0/4210084248 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a8df0640 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd7a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 -- 172.21.15.45:0/4210084248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd53310 msgr2=0x56021cd79e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd53310 0x56021cd79e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 -- 172.21.15.45:0/4210084248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 msgr2=0x56021cd7a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd7a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.785+0000 7f81a3fff640 1 -- 172.21.15.45:0/4210084248 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021cd4ee90 con 0x56021cd529f0 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81a3fff640 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd60f70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8194097d50 tx=0x7f81940d2f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c422a4266c063ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f8199ffb640 1 -- 172.21.15.45:0/4210084248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194012110 con 0x56021cd529f0 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f8199ffb640 1 -- 172.21.15.45:0/4210084248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819409bff0 con 0x56021cd529f0 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f8199ffb640 1 -- 172.21.15.45:0/4210084248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194099bf0 con 0x56021cd529f0 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 -- 172.21.15.45:0/4210084248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 msgr2=0x56021cd60f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd60f70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8194097d50 tx=0x7f81940d2f00 comp rx=0 tx=0).stop 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 -- 172.21.15.45:0/4210084248 shutdown_connections 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd620c0 0x56021cd7a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd53310 0x56021cd79e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/4210084248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd60f70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 -- 172.21.15.45:0/4210084248 >> 172.21.15.45:0/4210084248 conn(0x56021cd4f060 msgr2=0x56021cd60860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 -- 172.21.15.45:0/4210084248 shutdown_connections 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.786+0000 7f81ad5ec740 1 -- 172.21.15.45:0/4210084248 wait complete. 2024-09-10T10:10:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 -- start start 2024-09-10T10:10:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 Processor -- start 2024-09-10T10:10:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd66200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd53310 0x56021cd66740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cddbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41aa2d0 con 0x56021cd529f0 2024-09-10T10:10:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41ab740 con 0x56021cd53310 2024-09-10T10:10:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41aa9a0 con 0x56021cd620c0 2024-09-10T10:10:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81ad5ec740 10 client.0 _ll_register_callbacks cb 0x7f818c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd66200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd66200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58326/0 (socket says 172.21.15.45:58326) 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81a3fff640 1 -- 172.21.15.45:0/835450426 learned_addr learned my addr 172.21.15.45:0/835450426 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.787+0000 7f81a8df0640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cddbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f819bfff640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd53310 0x56021cd66740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81a3fff640 1 -- 172.21.15.45:0/835450426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 msgr2=0x56021cddbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81a3fff640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cddbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81a3fff640 1 -- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd53310 msgr2=0x56021cd66740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81a3fff640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd53310 0x56021cd66740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81a3fff640 1 -- 172.21.15.45:0/835450426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56021cd59d50 con 0x56021cd529f0 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f819bfff640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd53310 0x56021cd66740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:10:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81a3fff640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd66200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f819401af20 tx=0x7f819409b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d0f4599d7332fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81837fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81837fe640 1 -- 172.21.15.45:0/835450426 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819409fca0 con 0x56021cd529f0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81837fe640 1 -- 172.21.15.45:0/835450426 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7f8194018d60 con 0x56021cd529f0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819409bff0 con 0x56021cd529f0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81837fe640 1 -- 172.21.15.45:0/835450426 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f81940a4560 con 0x56021cd529f0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81ad5ec740 10 client.15591 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56021cddd2a0 con 0x56021cd529f0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81837fe640 1 -- 172.21.15.45:0/835450426 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819409bff0 con 0x56021cd529f0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81837fe640 1 -- 172.21.15.45:0/835450426 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f81940860b0 con 0x56021cd529f0 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81ad5ec740 10 client.15591 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81ad5ec740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81ad5ec740 20 client.15591 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.788+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cddd4d0 0x56021cddf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.789+0000 7f81ad5ec740 4 client.15591 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:10:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.789+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56021ccbed70 con 0x56021cddd4d0 2024-09-10T10:10:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.789+0000 7f819bfff640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cddd4d0 0x56021cddf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.789+0000 7f81837fe640 10 client.15591 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.789+0000 7f819bfff640 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cddd4d0 0x56021cddf8c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228728bc24eaa10f server_cookie=fcb4e15c1b9f4f97 in_seq=0 out_seq=0 2024-09-10T10:10:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.798+0000 7f81837fe640 1 -- 172.21.15.45:0/835450426 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56021ccbed70 con 0x56021cddd4d0 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.798+0000 7f81837fe640 10 client.15591 handle_command_reply: tid=0 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.798+0000 7f81ad5ec740 1 client.15591 shutdown 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 msgr2=0x56021cd66200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd66200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f819401af20 tx=0x7f819409b8f0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 shutdown_connections 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56021cddd4d0 0x56021cddf8c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56021cd620c0 0x56021cddbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56021cd53310 0x56021cd66740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 --2- 172.21.15.45:0/835450426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56021cd529f0 0x56021cd66200 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81837fe640 0 client.15591 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 >> 172.21.15.45:0/835450426 conn(0x56021cd4f060 msgr2=0x56021cd5f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 shutdown_connections 2024-09-10T10:10:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 1 -- 172.21.15.45:0/835450426 wait complete. 2024-09-10T10:10:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81ad5ec740 20 client.15591 trim_cache size 0 max 16384 2024-09-10T10:10:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:10:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:10:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:10:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f81900a2e20 msgr2=0x7f81900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81aa853640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f81900a2e20 0x7f81900a52e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f819c004ab0 tx=0x7f819c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4101ad0 msgr2=0x7f81a4198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.799+0000 7f81aa853640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4101ad0 0x7f81a4198230 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f819400c920 tx=0x7f819400cde0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 shutdown_connections 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f81900a2e20 0x7f81900a52e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81a4107b70 0x7f81a419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f81a4102470 0x7f81a4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 --2- 172.21.15.45:0/1036492017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81a4101ad0 0x7f81a4198230 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 >> 172.21.15.45:0/1036492017 conn(0x7f81a40fd7d0 msgr2=0x7f81a40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 shutdown_connections 2024-09-10T10:10:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:45.800+0000 7f81aa853640 1 -- 172.21.15.45:0/1036492017 wait complete. 2024-09-10T10:10:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:47 smithi162 ceph-mon[34963]: pgmap v233: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 853 B/s rd, 11 MiB/s wr, 99 op/s 2024-09-10T10:10:47.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:47 smithi045 ceph-mon[31114]: pgmap v233: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 853 B/s rd, 11 MiB/s wr, 99 op/s 2024-09-10T10:10:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:47 smithi047 ceph-mon[35244]: pgmap v233: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 853 B/s rd, 11 MiB/s wr, 99 op/s 2024-09-10T10:10:49.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:49 smithi162 ceph-mon[34963]: pgmap v234: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.3 MiB/s wr, 40 op/s 2024-09-10T10:10:49.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:49 smithi045 ceph-mon[31114]: pgmap v234: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.3 MiB/s wr, 40 op/s 2024-09-10T10:10:49.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:49 smithi047 ceph-mon[35244]: pgmap v234: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 4.3 MiB/s wr, 40 op/s 2024-09-10T10:10:51.653 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:51 smithi162 ceph-mon[34963]: pgmap v235: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:51.654 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:51 smithi045 ceph-mon[31114]: pgmap v235: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:51.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:51 smithi047 ceph-mon[35244]: pgmap v235: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:53.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:53 smithi162 ceph-mon[34963]: pgmap v236: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:53.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:53 smithi045 ceph-mon[31114]: pgmap v236: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:53.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:53 smithi047 ceph-mon[35244]: pgmap v236: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:55.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:55 smithi162 ceph-mon[34963]: pgmap v237: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-09-10T10:10:55.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:55 smithi045 ceph-mon[31114]: pgmap v237: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-09-10T10:10:55.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:55 smithi047 ceph-mon[35244]: pgmap v237: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-09-10T10:10:57.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:57 smithi162 ceph-mon[34963]: pgmap v238: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:57 smithi045 ceph-mon[31114]: pgmap v238: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:57.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:57 smithi047 ceph-mon[35244]: pgmap v238: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 -- 172.21.15.45:0/2478243308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc501069c0 msgr2=0x7efc50106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 --2- 172.21.15.45:0/2478243308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc501069c0 0x7efc50106da0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009930 tx=0x7efc4c02a420 comp rx=0 tx=0).stop 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 -- 172.21.15.45:0/2478243308 shutdown_connections 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 --2- 172.21.15.45:0/2478243308 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc50101350 0x7efc5010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 --2- 172.21.15.45:0/2478243308 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc501009b0 0x7efc50100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 --2- 172.21.15.45:0/2478243308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc501069c0 0x7efc50106da0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 -- 172.21.15.45:0/2478243308 >> 172.21.15.45:0/2478243308 conn(0x7efc50077990 msgr2=0x7efc500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 -- 172.21.15.45:0/2478243308 shutdown_connections 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.901+0000 7efc58227640 1 -- 172.21.15.45:0/2478243308 wait complete. 2024-09-10T10:10:58.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 Processor -- start 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 -- start start 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc501009b0 0x7efc501983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50101350 0x7efc50198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc501069c0 0x7efc5019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc50110e00 con 0x7efc50101350 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc50110c80 con 0x7efc501009b0 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc58227640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efc50110f80 con 0x7efc501069c0 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc557c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50101350 0x7efc50198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc557c2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50101350 0x7efc50198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41234/0 (socket says 172.21.15.45:41234) 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc557c2640 1 -- 172.21.15.45:0/4047888864 learned_addr learned my addr 172.21.15.45:0/4047888864 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc567c4640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc501069c0 0x7efc5019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.902+0000 7efc55fc3640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc501009b0 0x7efc501983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc557c2640 1 -- 172.21.15.45:0/4047888864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc501069c0 msgr2=0x7efc5019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc557c2640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc501069c0 0x7efc5019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc557c2640 1 -- 172.21.15.45:0/4047888864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc501009b0 msgr2=0x7efc501983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc557c2640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc501009b0 0x7efc501983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc557c2640 1 -- 172.21.15.45:0/4047888864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5019d3b0 con 0x7efc50101350 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc557c2640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50101350 0x7efc50198900 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efc4000d770 tx=0x7efc4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc46ffd640 1 -- 172.21.15.45:0/4047888864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40017070 con 0x7efc50101350 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc46ffd640 1 -- 172.21.15.45:0/4047888864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc40005500 con 0x7efc50101350 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5019d6a0 con 0x7efc50101350 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc46ffd640 1 -- 172.21.15.45:0/4047888864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40004e00 con 0x7efc50101350 2024-09-10T10:10:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.903+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc501a4f30 con 0x7efc50101350 2024-09-10T10:10:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc46ffd640 1 -- 172.21.15.45:0/4047888864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efc40004fa0 con 0x7efc50101350 2024-09-10T10:10:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 Processor -- start 2024-09-10T10:10:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 -- start start 2024-09-10T10:10:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20ae820 0x55fca20b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20b90c0 0x55fca20bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fca200e2f0 con 0x55fca20ae820 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fca1fb6270 con 0x55fca20b90c0 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.906+0000 7efc5afc1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fca200a2b0 con 0x55fca20adf00 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20ae820 0x55fca20b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20ae820 0x55fca20b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41238/0 (socket says 172.21.15.45:41238) 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 -- 172.21.15.45:0/2308957132 learned_addr learned my addr 172.21.15.45:0/2308957132 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc567c4640 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc557c2640 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20b90c0 0x55fca20bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 -- 172.21.15.45:0/2308957132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 msgr2=0x55fca20ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 -- 172.21.15.45:0/2308957132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20b90c0 msgr2=0x55fca20bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20b90c0 0x55fca20bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 -- 172.21.15.45:0/2308957132 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca1fb5340 con 0x55fca20ae820 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc46ffd640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efc3c0a2da0 0x7efc3c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc567c4640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efc3c0a2da0 0x7efc3c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc55fc3640 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20ae820 0x55fca20b8b80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efc4c02a940 tx=0x7efc4c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978d95bb428ea1b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc46ffd640 1 -- 172.21.15.45:0/4047888864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7efc400cee40 con 0x7efc50101350 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc3bfff640 1 -- 172.21.15.45:0/2308957132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c040070 con 0x55fca20ae820 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc3bfff640 1 -- 172.21.15.45:0/2308957132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4c0319f0 con 0x55fca20ae820 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.907+0000 7efc3bfff640 1 -- 172.21.15.45:0/2308957132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c035040 con 0x55fca20ae820 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 -- 172.21.15.45:0/2308957132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20ae820 msgr2=0x55fca20b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc567c4640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efc3c0a2da0 0x7efc3c0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc48005fd0 tx=0x7efc48009600 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20ae820 0x55fca20b8b80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efc4c02a940 tx=0x7efc4c005c00 comp rx=0 tx=0).stop 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 -- 172.21.15.45:0/2308957132 shutdown_connections 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20b90c0 0x55fca20bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20ae820 0x55fca20b8b80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2308957132 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 -- 172.21.15.45:0/2308957132 >> 172.21.15.45:0/2308957132 conn(0x55fca20aa470 msgr2=0x55fca20aa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 -- 172.21.15.45:0/2308957132 shutdown_connections 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.908+0000 7efc5afc1740 1 -- 172.21.15.45:0/2308957132 wait complete. 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 -- start start 2024-09-10T10:10:58.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 Processor -- start 2024-09-10T10:10:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20adf00 0x55fca20b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20ae820 0x55fca20b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fca20bbc90 con 0x55fca20b90c0 2024-09-10T10:10:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fca1f97290 con 0x55fca20adf00 2024-09-10T10:10:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fca20bbe10 con 0x55fca20ae820 2024-09-10T10:10:58.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc5afc1740 10 client.0 _ll_register_callbacks cb 0x55fca1fc8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.909+0000 7efc567c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41244/0 (socket says 172.21.15.45:41244) 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 -- 172.21.15.45:0/794875210 learned_addr learned my addr 172.21.15.45:0/794875210 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc55fc3640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20adf00 0x55fca20b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc557c2640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20ae820 0x55fca20b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 -- 172.21.15.45:0/794875210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20ae820 msgr2=0x55fca20b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20ae820 0x55fca20b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 -- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20adf00 msgr2=0x55fca20b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20adf00 0x55fca20b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 -- 172.21.15.45:0/794875210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca1d21210 con 0x55fca20b90c0 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc55fc3640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20adf00 0x55fca20b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:10:58.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc567c4640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20b68c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efc48011d40 tx=0x7efc48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2024d8b4258b644d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc397fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc397fa640 1 -- 172.21.15.45:0/794875210 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48004d20 con 0x55fca20b90c0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc397fa640 1 -- 172.21.15.45:0/794875210 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7efc48005020 con 0x55fca20b90c0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc4c0319f0 con 0x55fca20b90c0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.910+0000 7efc397fa640 1 -- 172.21.15.45:0/794875210 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efc48005260 con 0x55fca20b90c0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 10 client.15603 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fca1ded620 con 0x55fca20b90c0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc397fa640 1 -- 172.21.15.45:0/794875210 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4c0319f0 con 0x55fca20b90c0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc397fa640 1 -- 172.21.15.45:0/794875210 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efc4800f040 con 0x55fca20b90c0 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 10 client.15603 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:58.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 20 client.15603 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca20c8f00 0x55fca20cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 4 client.15603 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca201ba20 con 0x55fca20c8f00 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.911+0000 7efc55fc3640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca20c8f00 0x55fca20cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.912+0000 7efc397fa640 10 client.15603 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.912+0000 7efc55fc3640 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca20c8f00 0x55fca20cb330 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfcab9a12d9a1f1b server_cookie=18c12cf601d93c27 in_seq=0 out_seq=0 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.912+0000 7efc397fa640 1 -- 172.21.15.45:0/794875210 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fca201ba20 con 0x55fca20c8f00 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.912+0000 7efc397fa640 10 client.15603 handle_command_reply: tid=0 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.912+0000 7efc5afc1740 1 client.15603 shutdown 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 msgr2=0x55fca20b68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20b68c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efc48011d40 tx=0x7efc48004a40 comp rx=0 tx=0).stop 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 shutdown_connections 2024-09-10T10:10:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca20c8f00 0x55fca20cb330 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20b68c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20ae820 0x55fca20b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc397fa640 0 client.15603 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 --2- 172.21.15.45:0/794875210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20adf00 0x55fca20b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 >> 172.21.15.45:0/794875210 conn(0x55fca20aa470 msgr2=0x55fca20b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 shutdown_connections 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 1 -- 172.21.15.45:0/794875210 wait complete. 2024-09-10T10:10:58.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.913+0000 7efc5afc1740 20 client.15603 trim_cache size 0 max 16384 2024-09-10T10:10:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 Processor -- start 2024-09-10T10:10:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 -- start start 2024-09-10T10:10:58.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20adf00 0x55fca20d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20b90c0 0x55fca20b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc501a95e0 con 0x55fca20adf00 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc501abe40 con 0x55fca20ae820 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc5afc1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efc501a9c30 con 0x55fca20b90c0 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc55fc3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20adf00 0x55fca20d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc55fc3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20adf00 0x55fca20d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41260/0 (socket says 172.21.15.45:41260) 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.928+0000 7efc55fc3640 1 -- 172.21.15.45:0/4132215384 learned_addr learned my addr 172.21.15.45:0/4132215384 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc557c2640 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc567c4640 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20b90c0 0x55fca20b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc55fc3640 1 -- 172.21.15.45:0/4132215384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20b90c0 msgr2=0x55fca20b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc55fc3640 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20b90c0 0x55fca20b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc55fc3640 1 -- 172.21.15.45:0/4132215384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 msgr2=0x55fca20d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc55fc3640 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc55fc3640 1 -- 172.21.15.45:0/4132215384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca20ae650 con 0x55fca20adf00 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc55fc3640 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20adf00 0x55fca20d0c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efc4c005560 tx=0x7efc4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e9e8179b627e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc3b7fe640 1 -- 172.21.15.45:0/4132215384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c040040 con 0x55fca20adf00 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc3b7fe640 1 -- 172.21.15.45:0/4132215384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4c005600 con 0x55fca20adf00 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.929+0000 7efc3b7fe640 1 -- 172.21.15.45:0/4132215384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c035040 con 0x55fca20adf00 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 -- 172.21.15.45:0/4132215384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20adf00 msgr2=0x55fca20d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20adf00 0x55fca20d0c90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efc4c005560 tx=0x7efc4c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 -- 172.21.15.45:0/4132215384 shutdown_connections 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20b90c0 0x55fca20b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 --2- 172.21.15.45:0/4132215384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20adf00 0x55fca20d0c90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 -- 172.21.15.45:0/4132215384 >> 172.21.15.45:0/4132215384 conn(0x55fca20aa470 msgr2=0x55fca20b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 -- 172.21.15.45:0/4132215384 shutdown_connections 2024-09-10T10:10:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 -- 172.21.15.45:0/4132215384 wait complete. 2024-09-10T10:10:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 -- start start 2024-09-10T10:10:58.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 Processor -- start 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20c1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20b9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.930+0000 7efc5afc1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc5afc1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc501aa300 con 0x55fca20b90c0 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc5afc1740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efc501ab770 con 0x55fca20ae820 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc5afc1740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7efc501aa9d0 con 0x55fca20adf00 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc5afc1740 10 client.0 _ll_register_callbacks cb 0x7efc480119f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc557c2640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20b9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20ba480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41262/0 (socket says 172.21.15.45:41262) 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 -- 172.21.15.45:0/2732701120 learned_addr learned my addr 172.21.15.45:0/2732701120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:10:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc55fc3640 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 -- 172.21.15.45:0/2732701120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 msgr2=0x55fca20c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20c1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 -- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 msgr2=0x55fca20b9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20b9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 -- 172.21.15.45:0/2732701120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca20bacd0 con 0x55fca20b90c0 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc557c2640 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20b9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc567c4640 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20ba480 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efc48013ef0 tx=0x7efc48005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249b37798241de78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc397fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:10:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc397fa640 1 -- 172.21.15.45:0/2732701120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4803d060 con 0x55fca20b90c0 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc397fa640 1 -- 172.21.15.45:0/2732701120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8556+0+0 (secure 0 0 0) 0x7efc48003970 con 0x55fca20b90c0 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.931+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc4c005600 con 0x55fca20b90c0 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc397fa640 1 -- 172.21.15.45:0/2732701120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7efc48003b70 con 0x55fca20b90c0 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 10 client.15609 fetch_fsmap learned FSMap version 21 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fca2133cc0 con 0x55fca20b90c0 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc397fa640 1 -- 172.21.15.45:0/2732701120 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc4c005600 con 0x55fca20b90c0 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc397fa640 1 -- 172.21.15.45:0/2732701120 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7efc4800f040 con 0x55fca20b90c0 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 10 client.15609 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 20 client.15609 populate_metadata read hostname 'smithi045' 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca2134380 0x55fca2136770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 4 client.15609 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:10:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fca2015d70 con 0x55fca2134380 2024-09-10T10:10:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.932+0000 7efc55fc3640 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca2134380 0x55fca2136770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:10:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.933+0000 7efc397fa640 10 client.15609 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.933+0000 7efc55fc3640 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca2134380 0x55fca2136770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebb2611a6959ef9 server_cookie=6cdea243e5656b93 in_seq=0 out_seq=0 2024-09-10T10:10:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc397fa640 1 -- 172.21.15.45:0/2732701120 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fca2015d70 con 0x55fca2134380 2024-09-10T10:10:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc397fa640 10 client.15609 handle_command_reply: tid=0 2024-09-10T10:10:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 client.15609 shutdown 2024-09-10T10:10:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 msgr2=0x55fca20ba480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20ba480 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efc48013ef0 tx=0x7efc48005ce0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 shutdown_connections 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fca2134380 0x55fca2136770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fca20b90c0 0x55fca20ba480 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fca20ae820 0x55fca20b9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc397fa640 0 client.15609 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 --2- 172.21.15.45:0/2732701120 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fca20adf00 0x55fca20c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 >> 172.21.15.45:0/2732701120 conn(0x55fca20aa470 msgr2=0x55fca20a49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 shutdown_connections 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 1 -- 172.21.15.45:0/2732701120 wait complete. 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.940+0000 7efc5afc1740 20 client.15609 trim_cache size 0 max 16384 2024-09-10T10:10:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:10:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:10:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:10:58.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efc3c0a2da0 msgr2=0x7efc3c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efc3c0a2da0 0x7efc3c0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc48005fd0 tx=0x7efc48009600 comp rx=0 tx=0).stop 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50101350 msgr2=0x7efc50198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50101350 0x7efc50198900 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efc4000d770 tx=0x7efc4000dc30 comp rx=0 tx=0).stop 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 shutdown_connections 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7efc501069c0 0x7efc5019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc50101350 0x7efc50198900 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7efc3c0a2da0 0x7efc3c0a5260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 --2- 172.21.15.45:0/4047888864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efc501009b0 0x7efc501983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 >> 172.21.15.45:0/4047888864 conn(0x7efc50077990 msgr2=0x7efc50069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 shutdown_connections 2024-09-10T10:10:58.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:10:58.941+0000 7efc58227640 1 -- 172.21.15.45:0/4047888864 wait complete. 2024-09-10T10:10:59.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:10:59 smithi162 ceph-mon[34963]: pgmap v239: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:59.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:10:59 smithi045 ceph-mon[31114]: pgmap v239: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:10:59.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:10:59 smithi047 ceph-mon[35244]: pgmap v239: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-09-10T10:11:01.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:01 smithi162 ceph-mon[34963]: pgmap v240: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T10:11:01.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:01 smithi045 ceph-mon[31114]: pgmap v240: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T10:11:01.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:01 smithi047 ceph-mon[35244]: pgmap v240: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T10:11:03.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:03 smithi162 ceph-mon[34963]: pgmap v241: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:11:03.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:03 smithi045 ceph-mon[31114]: pgmap v241: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:11:03.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:03 smithi047 ceph-mon[35244]: pgmap v241: 97 pgs: 97 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:11:05.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:05 smithi162 ceph-mon[34963]: pgmap v242: 97 pgs: 97 active+clean; 61 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 118 op/s 2024-09-10T10:11:05.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:05 smithi045 ceph-mon[31114]: pgmap v242: 97 pgs: 97 active+clean; 61 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 118 op/s 2024-09-10T10:11:05.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:05 smithi047 ceph-mon[35244]: pgmap v242: 97 pgs: 97 active+clean; 61 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 118 op/s 2024-09-10T10:11:07.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:07 smithi162 ceph-mon[34963]: pgmap v243: 97 pgs: 97 active+clean; 61 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 185 op/s 2024-09-10T10:11:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:07 smithi045 ceph-mon[31114]: pgmap v243: 97 pgs: 97 active+clean; 61 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 185 op/s 2024-09-10T10:11:07.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:07 smithi047 ceph-mon[35244]: pgmap v243: 97 pgs: 97 active+clean; 61 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 18 MiB/s wr, 185 op/s 2024-09-10T10:11:09.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:09 smithi162 ceph-mon[34963]: pgmap v244: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 310 op/s 2024-09-10T10:11:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:09 smithi045 ceph-mon[31114]: pgmap v244: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 310 op/s 2024-09-10T10:11:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:09 smithi047 ceph-mon[35244]: pgmap v244: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 12 KiB/s rd, 32 MiB/s wr, 310 op/s 2024-09-10T10:11:11.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:11 smithi162 ceph-mon[34963]: pgmap v245: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 348 op/s 2024-09-10T10:11:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:11 smithi162 ceph-mon[34963]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:11:11.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:11 smithi045 ceph-mon[31114]: pgmap v245: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 348 op/s 2024-09-10T10:11:11.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:11 smithi045 ceph-mon[31114]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:11:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:11 smithi047 ceph-mon[35244]: pgmap v245: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 17 KiB/s rd, 36 MiB/s wr, 348 op/s 2024-09-10T10:11:11.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:11 smithi047 ceph-mon[35244]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T10:11:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 -- 172.21.15.45:0/4202882407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 msgr2=0x7fa77810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 --2- 172.21.15.45:0/4202882407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77810e560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa770009ca0 tx=0x7fa77002a950 comp rx=0 tx=0).stop 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 -- 172.21.15.45:0/4202882407 shutdown_connections 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 --2- 172.21.15.45:0/4202882407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77810e560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 --2- 172.21.15.45:0/4202882407 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa778103640 0x7fa77810bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 --2- 172.21.15.45:0/4202882407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa778102d20 0x7fa778103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 -- 172.21.15.45:0/4202882407 >> 172.21.15.45:0/4202882407 conn(0x7fa778077990 msgr2=0x7fa7780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 -- 172.21.15.45:0/4202882407 shutdown_connections 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.046+0000 7fa77dab7640 1 -- 172.21.15.45:0/4202882407 wait complete. 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 Processor -- start 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 -- start start 2024-09-10T10:11:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa778102d20 0x7fa778198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa778103640 0x7fa7781988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa778111ed0 con 0x7fa77810c0f0 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa778111d50 con 0x7fa778103640 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa77dab7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa778112050 con 0x7fa778102d20 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38510/0 (socket says 172.21.15.45:38510) 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 -- 172.21.15.45:0/2623812510 learned_addr learned my addr 172.21.15.45:0/2623812510 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa776ffd640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa778103640 0x7fa7781988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa7777fe640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa778102d20 0x7fa778198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 -- 172.21.15.45:0/2623812510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa778102d20 msgr2=0x7fa778198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa778102d20 0x7fa778198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 -- 172.21.15.45:0/2623812510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa778103640 msgr2=0x7fa7781988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa778103640 0x7fa7781988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.047+0000 7fa777fff640 1 -- 172.21.15.45:0/2623812510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77819d350 con 0x7fa77810c0f0 2024-09-10T10:11:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.048+0000 7fa777fff640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77819cc20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa770009dc0 tx=0x7fa770003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.048+0000 7fa774ff9640 1 -- 172.21.15.45:0/2623812510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770034040 con 0x7fa77810c0f0 2024-09-10T10:11:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.048+0000 7fa774ff9640 1 -- 172.21.15.45:0/2623812510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa770003e50 con 0x7fa77810c0f0 2024-09-10T10:11:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.048+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77819d5e0 con 0x7fa77810c0f0 2024-09-10T10:11:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.048+0000 7fa774ff9640 1 -- 172.21.15.45:0/2623812510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700046c0 con 0x7fa77810c0f0 2024-09-10T10:11:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.048+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781a4ed0 con 0x7fa77810c0f0 2024-09-10T10:11:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa774ff9640 1 -- 172.21.15.45:0/2623812510 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa770037020 con 0x7fa77810c0f0 2024-09-10T10:11:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 Processor -- start 2024-09-10T10:11:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 -- start start 2024-09-10T10:11:12.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1192182f0 con 0x55b1192b3990 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b1191c0270 con 0x55b1192b42b0 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa78084e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b1192142b0 con 0x55b1192c30c0 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38522/0 (socket says 172.21.15.45:38522) 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa776ffd640 1 -- 172.21.15.45:0/3356233747 learned_addr learned my addr 172.21.15.45:0/3356233747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa7777fe640 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.051+0000 7fa776ffd640 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa774ff9640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa7600a2e20 0x7fa7600a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa7777fe640 1 -- 172.21.15.45:0/3356233747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 msgr2=0x55b1192c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa7777fe640 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa7777fe640 1 -- 172.21.15.45:0/3356233747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 msgr2=0x55b1192b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa7777fe640 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa7777fe640 1 -- 172.21.15.45:0/3356233747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1191bf340 con 0x55b1192b42b0 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa776ffd640 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa777fff640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa7600a2e20 0x7fa7600a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa774ff9640 1 -- 172.21.15.45:0/2623812510 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8600+0+0 (secure 0 0 0) 0x7fa7700f01c0 con 0x7fa77810c0f0 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa7777fe640 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c2b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa768009c20 tx=0x7fa768011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96920b50c2041e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa75dffb640 1 -- 172.21.15.45:0/3356233747 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76801e070 con 0x55b1192b42b0 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa75dffb640 1 -- 172.21.15.45:0/3356233747 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768009dd0 con 0x55b1192b42b0 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.052+0000 7fa75dffb640 1 -- 172.21.15.45:0/3356233747 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680187d0 con 0x55b1192b42b0 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 -- 172.21.15.45:0/3356233747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 msgr2=0x55b1192c2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa777fff640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa7600a2e20 0x7fa7600a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7700bc440 tx=0x7fa77008f7b0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c2b80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa768009c20 tx=0x7fa768011810 comp rx=0 tx=0).stop 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 -- 172.21.15.45:0/3356233747 shutdown_connections 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c2b80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 --2- 172.21.15.45:0/3356233747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 -- 172.21.15.45:0/3356233747 >> 172.21.15.45:0/3356233747 conn(0x55b1192b0060 msgr2=0x55b1192b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 -- 172.21.15.45:0/3356233747 shutdown_connections 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 -- 172.21.15.45:0/3356233747 wait complete. 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 -- start start 2024-09-10T10:11:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.053+0000 7fa78084e740 1 Processor -- start 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b1192c5cf0 con 0x55b1192b3990 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b1191a1290 con 0x55b1192b42b0 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55b1192c5e70 con 0x55b1192c30c0 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 10 client.0 _ll_register_callbacks cb 0x55b1191d2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192c0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38538/0 (socket says 172.21.15.45:38538) 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa776ffd640 1 -- 172.21.15.45:0/970920499 learned_addr learned my addr 172.21.15.45:0/970920499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:12.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:34546/0 (socket says 172.21.15.45:34546) 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa776ffd640 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 -- 172.21.15.45:0/970920499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 msgr2=0x55b1192c0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 -- 172.21.15.45:0/970920499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 msgr2=0x55b1192c0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192c0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 -- 172.21.15.45:0/970920499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b118f2b210 con 0x55b1192b42b0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa7777fe640 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c0560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa76801c040 tx=0x7fa768019e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b1d1b205b55a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa756ffd640 1 -- 172.21.15.45:0/970920499 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76801e040 con 0x55b1192b42b0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.054+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa768009dd0 con 0x55b1192b42b0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa756ffd640 1 -- 172.21.15.45:0/970920499 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8600+0+0 (secure 0 0 0) 0x7fa768005050 con 0x55b1192b42b0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa756ffd640 1 -- 172.21.15.45:0/970920499 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa768031050 con 0x55b1192b42b0 2024-09-10T10:11:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 10 client.25250 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b118ff7620 con 0x55b1192b42b0 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa756ffd640 1 -- 172.21.15.45:0/970920499 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa768009dd0 con 0x55b1192b42b0 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa756ffd640 1 -- 172.21.15.45:0/970920499 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa76802c030 con 0x55b1192b42b0 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 10 client.25250 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 10 client.25250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 20 client.25250 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b1192d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 4 client.25250 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b119225a20 con 0x55b1192d3030 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.055+0000 7fa776ffd640 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b1192d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa756ffd640 10 client.25250 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa776ffd640 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b1192d5420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54fae620f488d6 server_cookie=20679ec6e4341d6a in_seq=0 out_seq=0 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa756ffd640 1 -- 172.21.15.45:0/970920499 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b119225a20 con 0x55b1192d3030 2024-09-10T10:11:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa756ffd640 10 client.25250 handle_command_reply: tid=0 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa78084e740 1 client.25250 shutdown 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 msgr2=0x55b1192c0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa78084e740 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c0560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa76801c040 tx=0x7fa768019e00 comp rx=0 tx=0).stop 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 shutdown_connections 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.056+0000 7fa78084e740 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b1192d5420 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa78084e740 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192c30c0 0x55b1192c0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa78084e740 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b42b0 0x55b1192c0560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa756ffd640 0 client.25250 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa78084e740 1 --2- 172.21.15.45:0/970920499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192b3990 0x55b1192c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 >> 172.21.15.45:0/970920499 conn(0x55b1192b0060 msgr2=0x55b1192b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 shutdown_connections 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa78084e740 1 -- 172.21.15.45:0/970920499 wait complete. 2024-09-10T10:11:12.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.057+0000 7fa78084e740 20 client.25250 trim_cache size 0 max 16384 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 Processor -- start 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 -- start start 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192dab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192db050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781a9640 con 0x55b1192c30c0 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781abe80 con 0x55b1192b3990 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa78084e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781a9c90 con 0x55b1192b42b0 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38554/0 (socket says 172.21.15.45:38554) 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa776ffd640 1 -- 172.21.15.45:0/4212174925 learned_addr learned my addr 172.21.15.45:0/4212174925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.071+0000 7fa7777fe640 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192db050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa776ffd640 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192dab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa776ffd640 1 -- 172.21.15.45:0/4212174925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 msgr2=0x55b1192db050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa776ffd640 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192db050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa776ffd640 1 -- 172.21.15.45:0/4212174925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 msgr2=0x55b1192dab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa776ffd640 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192dab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa776ffd640 1 -- 172.21.15.45:0/4212174925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1192befb0 con 0x55b1192c30c0 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa776ffd640 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bd280 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00c010 tx=0x7fa76c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a96a86de7beaf60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa75d7fa640 1 -- 172.21.15.45:0/4212174925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c005020 con 0x55b1192c30c0 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa75d7fa640 1 -- 172.21.15.45:0/4212174925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76c0047a0 con 0x55b1192c30c0 2024-09-10T10:11:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.072+0000 7fa75d7fa640 1 -- 172.21.15.45:0/4212174925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0127e0 con 0x55b1192c30c0 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- 172.21.15.45:0/4212174925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 msgr2=0x55b1192bd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bd280 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00c010 tx=0x7fa76c00b6b0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- 172.21.15.45:0/4212174925 shutdown_connections 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bd280 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192db050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 --2- 172.21.15.45:0/4212174925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192dab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- 172.21.15.45:0/4212174925 >> 172.21.15.45:0/4212174925 conn(0x55b1192b0060 msgr2=0x55b1192af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- 172.21.15.45:0/4212174925 shutdown_connections 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- 172.21.15.45:0/4212174925 wait complete. 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- start start 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 Processor -- start 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781aa340 con 0x55b1192c30c0 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781ab7b0 con 0x55b1192b3990 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781aaa10 con 0x55b1192b42b0 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa78084e740 10 client.0 _ll_register_callbacks cb 0x7fa76801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa776ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38558/0 (socket says 172.21.15.45:38558) 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.073+0000 7fa776ffd640 1 -- 172.21.15.45:0/2174080431 learned_addr learned my addr 172.21.15.45:0/2174080431 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa7777fe640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa776ffd640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa776ffd640 1 -- 172.21.15.45:0/2174080431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 msgr2=0x55b1192bd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa776ffd640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192bd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa776ffd640 1 -- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 msgr2=0x55b1192bd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa776ffd640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192bd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa776ffd640 1 -- 172.21.15.45:0/2174080431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1192b40e0 con 0x55b1192c30c0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa7777fe640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa776ffd640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bdac0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa76c004570 tx=0x7fa76c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448bf779f01a442d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa756ffd640 1 -- 172.21.15.45:0/2174080431 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c004f00 con 0x55b1192c30c0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa76c0047a0 con 0x55b1192c30c0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa756ffd640 1 -- 172.21.15.45:0/2174080431 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8600+0+0 (secure 0 0 0) 0x7fa76c0050a0 con 0x55b1192c30c0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa756ffd640 1 -- 172.21.15.45:0/2174080431 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa76c0052a0 con 0x55b1192c30c0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa78084e740 10 client.15627 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b11933dc50 con 0x55b1192c30c0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.074+0000 7fa756ffd640 1 -- 172.21.15.45:0/2174080431 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa76c0047a0 con 0x55b1192c30c0 2024-09-10T10:11:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa756ffd640 1 -- 172.21.15.45:0/2174080431 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa76c03d020 con 0x55b1192c30c0 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa78084e740 10 client.15627 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa78084e740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa78084e740 20 client.15627 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa78084e740 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b119340280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa78084e740 4 client.15627 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b11921fd70 con 0x55b1192d3030 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa7777fe640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b119340280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa756ffd640 10 client.15627 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.075+0000 7fa7777fe640 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b119340280 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152db2520175d914 server_cookie=ff9c5810e446003e in_seq=0 out_seq=0 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.088+0000 7fa756ffd640 1 -- 172.21.15.45:0/2174080431 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b11921fd70 con 0x55b1192d3030 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa756ffd640 10 client.15627 handle_command_reply: tid=0 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 client.15627 shutdown 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 msgr2=0x55b1192bdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bdac0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa76c004570 tx=0x7fa76c00be00 comp rx=0 tx=0).stop 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 shutdown_connections 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55b1192d3030 0x55b119340280 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b1192c30c0 0x55b1192bdac0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55b1192b42b0 0x55b1192bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 --2- 172.21.15.45:0/2174080431 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b1192b3990 0x55b1192bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa756ffd640 0 client.15627 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 >> 172.21.15.45:0/2174080431 conn(0x55b1192b0060 msgr2=0x55b1192bffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 shutdown_connections 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 1 -- 172.21.15.45:0/2174080431 wait complete. 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa78084e740 20 client.15627 trim_cache size 0 max 16384 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:11:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:11:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa7600a2e20 msgr2=0x7fa7600a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa77dab7640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa7600a2e20 0x7fa7600a52e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7700bc440 tx=0x7fa77008f7b0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 msgr2=0x7fa77819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa77dab7640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77819cc20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa770009dc0 tx=0x7fa770003ce0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 shutdown_connections 2024-09-10T10:11:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa77dab7640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa77810c0f0 0x7fa77819cc20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.089+0000 7fa77dab7640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa7600a2e20 0x7fa7600a52e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.090+0000 7fa77dab7640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa778103640 0x7fa7781988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.090+0000 7fa77dab7640 1 --2- 172.21.15.45:0/2623812510 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa778102d20 0x7fa778198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.090+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 >> 172.21.15.45:0/2623812510 conn(0x7fa778077990 msgr2=0x7fa7780fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.090+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 shutdown_connections 2024-09-10T10:11:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:12.090+0000 7fa77dab7640 1 -- 172.21.15.45:0/2623812510 wait complete. 2024-09-10T10:11:13.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:13 smithi162 ceph-mon[34963]: pgmap v247: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 20 KiB/s rd, 45 MiB/s wr, 434 op/s 2024-09-10T10:11:13.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:13 smithi045 ceph-mon[31114]: pgmap v247: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 20 KiB/s rd, 45 MiB/s wr, 434 op/s 2024-09-10T10:11:13.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:13 smithi047 ceph-mon[35244]: pgmap v247: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 20 KiB/s rd, 45 MiB/s wr, 434 op/s 2024-09-10T10:11:15.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:15 smithi162 ceph-mon[34963]: pgmap v248: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 53 GiB data, 172 GiB used, 900 GiB / 1.0 TiB avail; 14 KiB/s rd, 38 MiB/s wr, 350 op/s 2024-09-10T10:11:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:15 smithi045 ceph-mon[31114]: pgmap v248: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 53 GiB data, 172 GiB used, 900 GiB / 1.0 TiB avail; 14 KiB/s rd, 38 MiB/s wr, 350 op/s 2024-09-10T10:11:15.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:15 smithi047 ceph-mon[35244]: pgmap v248: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 53 GiB data, 172 GiB used, 900 GiB / 1.0 TiB avail; 14 KiB/s rd, 38 MiB/s wr, 350 op/s 2024-09-10T10:11:17.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:17 smithi162 ceph-mon[34963]: pgmap v249: 97 pgs: 97 active+clean; 51 GiB data, 167 GiB used, 905 GiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 268 op/s 2024-09-10T10:11:17.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:17 smithi045 ceph-mon[31114]: pgmap v249: 97 pgs: 97 active+clean; 51 GiB data, 167 GiB used, 905 GiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 268 op/s 2024-09-10T10:11:17.661 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:17 smithi047 ceph-mon[35244]: pgmap v249: 97 pgs: 97 active+clean; 51 GiB data, 167 GiB used, 905 GiB / 1.0 TiB avail; 14 KiB/s rd, 29 MiB/s wr, 268 op/s 2024-09-10T10:11:19.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:19 smithi162 ceph-mon[34963]: pgmap v250: 97 pgs: 97 active+clean; 51 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 118 op/s 2024-09-10T10:11:19.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:19 smithi045 ceph-mon[31114]: pgmap v250: 97 pgs: 97 active+clean; 51 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 118 op/s 2024-09-10T10:11:19.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:19 smithi047 ceph-mon[35244]: pgmap v250: 97 pgs: 97 active+clean; 51 GiB data, 166 GiB used, 907 GiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 118 op/s 2024-09-10T10:11:21.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:21 smithi162 ceph-mon[34963]: pgmap v251: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.4 MiB/s wr, 74 op/s 2024-09-10T10:11:21.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:21 smithi162 ceph-mon[34963]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:11:21.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:21 smithi045 ceph-mon[31114]: pgmap v251: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.4 MiB/s wr, 74 op/s 2024-09-10T10:11:21.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:21 smithi045 ceph-mon[31114]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:11:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:21 smithi047 ceph-mon[35244]: pgmap v251: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.4 MiB/s wr, 74 op/s 2024-09-10T10:11:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:21 smithi047 ceph-mon[35244]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T10:11:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:23 smithi162 ceph-mon[34963]: pgmap v253: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-09-10T10:11:23.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:23 smithi045 ceph-mon[31114]: pgmap v253: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-09-10T10:11:23.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:23 smithi047 ceph-mon[35244]: pgmap v253: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.6 MiB/s wr, 56 op/s 2024-09-10T10:11:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.190+0000 7ff0dfd86640 1 -- 172.21.15.45:0/1612682709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8101ad0 msgr2=0x7ff0d8101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.190+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/1612682709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8101ad0 0x7ff0d8101f30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009930 tx=0x7ff0c802a420 comp rx=0 tx=0).stop 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 -- 172.21.15.45:0/1612682709 shutdown_connections 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/1612682709 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d8102470 0x7ff0d810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/1612682709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8101ad0 0x7ff0d8101f30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/1612682709 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d8107b70 0x7ff0d8107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 -- 172.21.15.45:0/1612682709 >> 172.21.15.45:0/1612682709 conn(0x7ff0d80fd7d0 msgr2=0x7ff0d80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 -- 172.21.15.45:0/1612682709 shutdown_connections 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 -- 172.21.15.45:0/1612682709 wait complete. 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 Processor -- start 2024-09-10T10:11:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.191+0000 7ff0dfd86640 1 -- start start 2024-09-10T10:11:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dfd86640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d8101ad0 0x7ff0d8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dfd86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102470 0x7ff0d8198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dfd86640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d8107b70 0x7ff0d819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dfd86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8111ee0 con 0x7ff0d8102470 2024-09-10T10:11:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dfd86640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8111d60 con 0x7ff0d8107b70 2024-09-10T10:11:25.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dfd86640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8112060 con 0x7ff0d8101ad0 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102470 0x7ff0d8198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102470 0x7ff0d8198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59370/0 (socket says 172.21.15.45:59370) 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 -- 172.21.15.45:0/3051774776 learned_addr learned my addr 172.21.15.45:0/3051774776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d8101ad0 0x7ff0d8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0de323640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d8107b70 0x7ff0d819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 -- 172.21.15.45:0/3051774776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d8101ad0 msgr2=0x7ff0d8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d8101ad0 0x7ff0d8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 -- 172.21.15.45:0/3051774776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d8107b70 msgr2=0x7ff0d819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d8107b70 0x7ff0d819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 -- 172.21.15.45:0/3051774776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d819d310 con 0x7ff0d8102470 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0de323640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d8107b70 0x7ff0d819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:11:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0dd321640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102470 0x7ff0d8198860 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009230 tx=0x7ff0c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0ceffd640 1 -- 172.21.15.45:0/3051774776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c803f030 con 0x7ff0d8102470 2024-09-10T10:11:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.192+0000 7ff0ceffd640 1 -- 172.21.15.45:0/3051774776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c8003ad0 con 0x7ff0d8102470 2024-09-10T10:11:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.193+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d819d5a0 con 0x7ff0d8102470 2024-09-10T10:11:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.193+0000 7ff0ceffd640 1 -- 172.21.15.45:0/3051774776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8035070 con 0x7ff0d8102470 2024-09-10T10:11:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.193+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d81a4e90 con 0x7ff0d8102470 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.195+0000 7ff0ceffd640 1 -- 172.21.15.45:0/3051774776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff0c8042030 con 0x7ff0d8102470 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.195+0000 7ff0e2bfd740 1 Processor -- start 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0e2bfd740 1 -- start start 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb1539fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153a02d0 0x55cb153aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153af0c0 0x55cb153b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cb153042f0 con 0x55cb153a02d0 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cb152ac270 con 0x55cb153af0c0 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55cb153002b0 con 0x55cb1539f9b0 2024-09-10T10:11:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0ddb22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153a02d0 0x55cb153aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0ddb22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153a02d0 0x55cb153aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59380/0 (socket says 172.21.15.45:59380) 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0ddb22640 1 -- 172.21.15.45:0/3744823084 learned_addr learned my addr 172.21.15.45:0/3744823084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0ceffd640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff0c40a2e20 0x7ff0c40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0dd321640 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153af0c0 0x55cb153b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0de323640 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb1539fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0de323640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff0c40a2e20 0x7ff0c40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.196+0000 7ff0ddb22640 1 -- 172.21.15.45:0/3744823084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 msgr2=0x55cb1539fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb1539fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0ddb22640 1 -- 172.21.15.45:0/3744823084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153af0c0 msgr2=0x55cb153b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153af0c0 0x55cb153b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0ddb22640 1 -- 172.21.15.45:0/3744823084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb152ab340 con 0x55cb153a02d0 2024-09-10T10:11:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0de323640 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb1539fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0ceffd640 1 -- 172.21.15.45:0/3051774776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff0c80ef570 con 0x7ff0d8102470 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153a02d0 0x55cb153aeb80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400bc50 tx=0x7ff0d40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3cfa479bb4470b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0c3fff640 1 -- 172.21.15.45:0/3744823084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d401c480 con 0x55cb153a02d0 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0de323640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff0c40a2e20 0x7ff0c40a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005c40 tx=0x7ff0d00094a0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0c3fff640 1 -- 172.21.15.45:0/3744823084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d401cb00 con 0x55cb153a02d0 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.197+0000 7ff0c3fff640 1 -- 172.21.15.45:0/3744823084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4011b90 con 0x55cb153a02d0 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/3744823084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153a02d0 msgr2=0x55cb153aeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153a02d0 0x55cb153aeb80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400bc50 tx=0x7ff0d40107a0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/3744823084 shutdown_connections 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153af0c0 0x55cb153b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153a02d0 0x55cb153aeb80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/3744823084 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb1539fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/3744823084 >> 172.21.15.45:0/3744823084 conn(0x55cb1539c060 msgr2=0x55cb1539c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/3744823084 shutdown_connections 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/3744823084 wait complete. 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 -- start start 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.198+0000 7ff0e2bfd740 1 Processor -- start 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb153ac060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cb153b1c90 con 0x55cb153af0c0 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55cb1528d290 con 0x55cb153a02d0 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55cb153b1e10 con 0x55cb1539f9b0 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0e2bfd740 10 client.0 _ll_register_callbacks cb 0x55cb152be140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153acae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59388/0 (socket says 172.21.15.45:59388) 2024-09-10T10:11:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 -- 172.21.15.45:0/1320999115 learned_addr learned my addr 172.21.15.45:0/1320999115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0dd321640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb153ac060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 -- 172.21.15.45:0/1320999115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 msgr2=0x55cb153ac060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb153ac060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 -- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 msgr2=0x55cb153ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0de323640 1 -- 172.21.15.45:0/1320999115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb15017210 con 0x55cb153af0c0 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.199+0000 7ff0dd321640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0de323640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153acae0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0010970 tx=0x7ff0d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99863c941201260 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1320999115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d002a060 con 0x55cb153af0c0 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d401cb00 con 0x55cb153af0c0 2024-09-10T10:11:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1320999115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff0d001db90 con 0x55cb153af0c0 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1320999115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff0d001dd90 con 0x55cb153af0c0 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0e2bfd740 10 client.15639 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cb150e3620 con 0x55cb153af0c0 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1320999115 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d401cb00 con 0x55cb153af0c0 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.200+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1320999115 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff0d0019040 con 0x55cb153af0c0 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0e2bfd740 10 client.15639 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0e2bfd740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0e2bfd740 20 client.15639 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb153bf120 0x55cb153c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0e2bfd740 4 client.15639 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb15311a20 con 0x55cb153bf120 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb153bf120 0x55cb153c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb153bf120 0x55cb153c1550 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19437a32550985aa server_cookie=aed9a8036be71c0e in_seq=0 out_seq=0 2024-09-10T10:11:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.201+0000 7ff0c17fa640 10 client.15639 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1320999115 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb15311a20 con 0x55cb153bf120 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0c17fa640 10 client.15639 handle_command_reply: tid=0 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 client.15639 shutdown 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 msgr2=0x55cb153acae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153acae0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0010970 tx=0x7ff0d0004ab0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 shutdown_connections 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb153bf120 0x55cb153c1550 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153acae0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0c17fa640 0 client.15639 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1320999115 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb1539f9b0 0x55cb153ac060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.202+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 >> 172.21.15.45:0/1320999115 conn(0x55cb1539c060 msgr2=0x55cb153a1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.203+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 shutdown_connections 2024-09-10T10:11:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.203+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1320999115 wait complete. 2024-09-10T10:11:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.203+0000 7ff0e2bfd740 20 client.15639 trim_cache size 0 max 16384 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.217+0000 7ff0e2bfd740 1 Processor -- start 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.217+0000 7ff0e2bfd740 1 -- start start 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.217+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb1539f9b0 0x55cb153bb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153af0c0 0x55cb153c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81a95f0 con 0x55cb1539f9b0 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81abe50 con 0x55cb153a02d0 2024-09-10T10:11:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81a9c40 con 0x55cb153af0c0 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb1539f9b0 0x55cb153bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb1539f9b0 0x55cb153bb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59396/0 (socket says 172.21.15.45:59396) 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 -- 172.21.15.45:0/2428593861 learned_addr learned my addr 172.21.15.45:0/2428593861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0de323640 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153af0c0 0x55cb153c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0dd321640 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 -- 172.21.15.45:0/2428593861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153af0c0 msgr2=0x55cb153c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153af0c0 0x55cb153c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 -- 172.21.15.45:0/2428593861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 msgr2=0x55cb153c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 -- 172.21.15.45:0/2428593861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb153aaff0 con 0x55cb1539f9b0 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0dd321640 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:11:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb1539f9b0 0x55cb153bb700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff0d401e010 tx=0x7ff0d4011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698102e803c94387 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0c37fe640 1 -- 172.21.15.45:0/2428593861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4005240 con 0x55cb1539f9b0 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0c37fe640 1 -- 172.21.15.45:0/2428593861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d401c960 con 0x55cb1539f9b0 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.218+0000 7ff0c37fe640 1 -- 172.21.15.45:0/2428593861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4011b20 con 0x55cb1539f9b0 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/2428593861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb1539f9b0 msgr2=0x55cb153bb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb1539f9b0 0x55cb153bb700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff0d401e010 tx=0x7ff0d4011650 comp rx=0 tx=0).stop 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/2428593861 shutdown_connections 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153af0c0 0x55cb153c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb153a02d0 0x55cb153c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/2428593861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb1539f9b0 0x55cb153bb700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/2428593861 >> 172.21.15.45:0/2428593861 conn(0x55cb1539c060 msgr2=0x55cb153ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/2428593861 shutdown_connections 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/2428593861 wait complete. 2024-09-10T10:11:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.219+0000 7ff0e2bfd740 1 -- start start 2024-09-10T10:11:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 1 Processor -- start 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb1539f9b0 0x55cb153b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153a02d0 0x55cb153b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81aa310 con 0x55cb153af0c0 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81ab780 con 0x55cb1539f9b0 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81aa9e0 con 0x55cb153a02d0 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0e2bfd740 10 client.0 _ll_register_callbacks cb 0x7ff0d00098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0de323640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0ddb22640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb1539f9b0 0x55cb153b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0de323640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59406/0 (socket says 172.21.15.45:59406) 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0de323640 1 -- 172.21.15.45:0/1703923753 learned_addr learned my addr 172.21.15.45:0/1703923753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.220+0000 7ff0dd321640 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153a02d0 0x55cb153b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0de323640 1 -- 172.21.15.45:0/1703923753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153a02d0 msgr2=0x55cb153b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0de323640 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153a02d0 0x55cb153b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0de323640 1 -- 172.21.15.45:0/1703923753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb1539f9b0 msgr2=0x55cb153b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0de323640 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb1539f9b0 0x55cb153b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0de323640 1 -- 172.21.15.45:0/1703923753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb15428ee0 con 0x55cb153af0c0 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0dd321640 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153a02d0 0x55cb153b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0de323640 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153b3920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000f80 tx=0x7ff0d001bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d661cbda072430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1703923753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d003c060 con 0x55cb153af0c0 2024-09-10T10:11:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1703923753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7ff0d0009550 con 0x55cb153af0c0 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d401c960 con 0x55cb153af0c0 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1703923753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff0d001d7d0 con 0x55cb153af0c0 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0e2bfd740 10 client.15645 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55cb1542a140 con 0x55cb153af0c0 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.221+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1703923753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d401c960 con 0x55cb153af0c0 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1703923753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff0d0019040 con 0x55cb153af0c0 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0e2bfd740 10 client.15645 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0e2bfd740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0e2bfd740 20 client.15645 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb1542a800 0x55cb1542cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0e2bfd740 4 client.15645 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:11:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb1530bd70 con 0x55cb1542a800 2024-09-10T10:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb1542a800 0x55cb1542cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0ddb22640 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb1542a800 0x55cb1542cc20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ded7336e8bf6b6 server_cookie=e4df1da654ae40e2 in_seq=0 out_seq=0 2024-09-10T10:11:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.222+0000 7ff0c17fa640 10 client.15645 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.227+0000 7ff0c17fa640 1 -- 172.21.15.45:0/1703923753 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb1530bd70 con 0x55cb1542a800 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.227+0000 7ff0c17fa640 10 client.15645 handle_command_reply: tid=0 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.227+0000 7ff0e2bfd740 1 client.15645 shutdown 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.227+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 msgr2=0x55cb153b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.227+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153b3920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000f80 tx=0x7ff0d001bbf0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 shutdown_connections 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55cb1542a800 0x55cb1542cc20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb153af0c0 0x55cb153b3920 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55cb153a02d0 0x55cb153b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 --2- 172.21.15.45:0/1703923753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55cb1539f9b0 0x55cb153b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0c17fa640 0 client.15645 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 >> 172.21.15.45:0/1703923753 conn(0x55cb1539c060 msgr2=0x55cb1539a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 shutdown_connections 2024-09-10T10:11:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 1 -- 172.21.15.45:0/1703923753 wait complete. 2024-09-10T10:11:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0e2bfd740 20 client.15645 trim_cache size 0 max 16384 2024-09-10T10:11:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:11:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:11:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:11:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff0c40a2e20 msgr2=0x7ff0c40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff0c40a2e20 0x7ff0c40a52e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005c40 tx=0x7ff0d00094a0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102470 msgr2=0x7ff0d8198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102470 0x7ff0d8198860 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009230 tx=0x7ff0c8004270 comp rx=0 tx=0).stop 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 shutdown_connections 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff0c40a2e20 0x7ff0c40a52e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d8107b70 0x7ff0d819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d8102470 0x7ff0d8198860 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 --2- 172.21.15.45:0/3051774776 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff0d8101ad0 0x7ff0d8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 >> 172.21.15.45:0/3051774776 conn(0x7ff0d80fd7d0 msgr2=0x7ff0d80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.228+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 shutdown_connections 2024-09-10T10:11:25.237 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:25.229+0000 7ff0dfd86640 1 -- 172.21.15.45:0/3051774776 wait complete. 2024-09-10T10:11:25.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:25 smithi162 ceph-mon[34963]: pgmap v254: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:11:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:25 smithi045 ceph-mon[31114]: pgmap v254: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:11:25.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:25 smithi047 ceph-mon[35244]: pgmap v254: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:11:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:27 smithi162 ceph-mon[34963]: pgmap v255: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:11:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:11:27.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:27 smithi045 ceph-mon[31114]: pgmap v255: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:11:27.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:11:27.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:27 smithi047 ceph-mon[35244]: pgmap v255: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:11:27.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:11:28.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:28 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:11:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:28 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:11:28.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:28 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:11:28.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:28 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:11:28.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:28 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:11:28.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:28 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:11:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:28 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:11:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:28 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:11:28.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:28 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:11:30.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:29 smithi162 ceph-mon[34963]: pgmap v256: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-09-10T10:11:30.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:29 smithi045 ceph-mon[31114]: pgmap v256: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-09-10T10:11:30.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:29 smithi047 ceph-mon[35244]: pgmap v256: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-09-10T10:11:32.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:31 smithi162 ceph-mon[34963]: pgmap v257: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:11:32.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:31 smithi045 ceph-mon[31114]: pgmap v257: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:11:32.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:31 smithi047 ceph-mon[35244]: pgmap v257: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:11:34.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:33 smithi162 ceph-mon[34963]: pgmap v258: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 284 B/s wr, 2 op/s 2024-09-10T10:11:34.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:33 smithi045 ceph-mon[31114]: pgmap v258: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 284 B/s wr, 2 op/s 2024-09-10T10:11:34.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:33 smithi047 ceph-mon[35244]: pgmap v258: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 284 B/s wr, 2 op/s 2024-09-10T10:11:36.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:35 smithi162 ceph-mon[34963]: pgmap v259: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:36.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:35 smithi045 ceph-mon[31114]: pgmap v259: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:36.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:35 smithi047 ceph-mon[35244]: pgmap v259: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:38.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:37 smithi162 ceph-mon[34963]: pgmap v260: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:38.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:37 smithi045 ceph-mon[31114]: pgmap v260: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:38.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:37 smithi047 ceph-mon[35244]: pgmap v260: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 -- 172.21.15.45:0/1022192638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 msgr2=0x7f241010d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 --2- 172.21.15.45:0/1022192638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f241010d8c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2408009cd0 tx=0x7f240802a8e0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 -- 172.21.15.45:0/1022192638 shutdown_connections 2024-09-10T10:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 --2- 172.21.15.45:0/1022192638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f241010d8c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 --2- 172.21.15.45:0/1022192638 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24101009f0 0x7f2410100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 --2- 172.21.15.45:0/1022192638 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2410106a00 0x7f2410106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 -- 172.21.15.45:0/1022192638 >> 172.21.15.45:0/1022192638 conn(0x7f24100fc6b0 msgr2=0x7f24100fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 -- 172.21.15.45:0/1022192638 shutdown_connections 2024-09-10T10:11:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.329+0000 7f2418445640 1 -- 172.21.15.45:0/1022192638 wait complete. 2024-09-10T10:11:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 Processor -- start 2024-09-10T10:11:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 -- start start 2024-09-10T10:11:38.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24101009f0 0x7f24100fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f24100fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2410106a00 0x7f24100fd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2410110e40 con 0x7f2410101390 2024-09-10T10:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2410110cc0 con 0x7f2410106a00 2024-09-10T10:11:38.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f2418445640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2410110fc0 con 0x7f24101009f0 2024-09-10T10:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f24159e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f24100fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f24159e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f24100fd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45308/0 (socket says 172.21.15.45:45308) 2024-09-10T10:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f24159e0640 1 -- 172.21.15.45:0/2298173274 learned_addr learned my addr 172.21.15.45:0/2298173274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.330+0000 7f24169e2640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2410106a00 0x7f24100fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24161e1640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24101009f0 0x7f24100fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24159e0640 1 -- 172.21.15.45:0/2298173274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24101009f0 msgr2=0x7f24100fcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24159e0640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24101009f0 0x7f24100fcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24159e0640 1 -- 172.21.15.45:0/2298173274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2410106a00 msgr2=0x7f24100fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24159e0640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2410106a00 0x7f24100fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24159e0640 1 -- 172.21.15.45:0/2298173274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101a6860 con 0x7f2410101390 2024-09-10T10:11:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24159e0640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f24100fd140 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f240000d950 tx=0x7f240000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24077fe640 1 -- 172.21.15.45:0/2298173274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400017030 con 0x7f2410101390 2024-09-10T10:11:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24077fe640 1 -- 172.21.15.45:0/2298173274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2400004020 con 0x7f2410101390 2024-09-10T10:11:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101a6b50 con 0x7f2410101390 2024-09-10T10:11:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f24077fe640 1 -- 172.21.15.45:0/2298173274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000047a0 con 0x7f2410101390 2024-09-10T10:11:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.331+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101a70a0 con 0x7f2410101390 2024-09-10T10:11:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.334+0000 7f24077fe640 1 -- 172.21.15.45:0/2298173274 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f24000041d0 con 0x7f2410101390 2024-09-10T10:11:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 Processor -- start 2024-09-10T10:11:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 -- start start 2024-09-10T10:11:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b92533ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b925410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b924982f0 con 0x562b9253ecb0 2024-09-10T10:11:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562b92440270 con 0x562b92534410 2024-09-10T10:11:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f241b1da740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562b924942b0 con 0x562b92533af0 2024-09-10T10:11:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24077fe640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23fc0a2da0 0x7f23fc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24159e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b925410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24161e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24169e2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b92533ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24161e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:36448/0 (socket says 172.21.15.45:36448) 2024-09-10T10:11:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24159e0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b925410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45324/0 (socket says 172.21.15.45:45324) 2024-09-10T10:11:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24161e1640 1 -- 172.21.15.45:0/1366172331 learned_addr learned my addr 172.21.15.45:0/1366172331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.335+0000 7f24169e2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b92533ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:43056/0 (socket says 172.21.15.45:43056) 2024-09-10T10:11:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24169e2640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23fc0a2da0 0x7f23fc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24077fe640 1 -- 172.21.15.45:0/2298173274 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f24000cee30 con 0x7f2410101390 2024-09-10T10:11:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24161e1640 1 -- 172.21.15.45:0/1366172331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 msgr2=0x562b92533ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24161e1640 1 --2- 172.21.15.45:0/1366172331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b92533ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24161e1640 1 -- 172.21.15.45:0/1366172331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 msgr2=0x562b925410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24161e1640 1 --2- 172.21.15.45:0/1366172331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b925410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24161e1640 1 -- 172.21.15.45:0/1366172331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9243f340 con 0x562b92534410 2024-09-10T10:11:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24161e1640 1 --2- 172.21.15.45:0/1366172331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253e770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f240c00dc30 tx=0x7f240c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f421d07de1c89d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f24169e2640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23fc0a2da0 0x7f23fc0a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2408004ab0 tx=0x7f2408004a40 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f2404bfe640 1 -- 172.21.15.45:0/1366172331 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c011440 con 0x562b92534410 2024-09-10T10:11:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f2404bfe640 1 -- 172.21.15.45:0/1366172331 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240c017070 con 0x562b92534410 2024-09-10T10:11:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.336+0000 7f2404bfe640 1 -- 172.21.15.45:0/1366172331 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c010930 con 0x562b92534410 2024-09-10T10:11:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 -- 172.21.15.45:0/1366172331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 msgr2=0x562b9253e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 --2- 172.21.15.45:0/1366172331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253e770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f240c00dc30 tx=0x7f240c00f730 comp rx=0 tx=0).stop 2024-09-10T10:11:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 -- 172.21.15.45:0/1366172331 shutdown_connections 2024-09-10T10:11:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 --2- 172.21.15.45:0/1366172331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b925410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 --2- 172.21.15.45:0/1366172331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253e770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 --2- 172.21.15.45:0/1366172331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b92533ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 -- 172.21.15.45:0/1366172331 >> 172.21.15.45:0/1366172331 conn(0x562b92530060 msgr2=0x562b92530440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 -- 172.21.15.45:0/1366172331 shutdown_connections 2024-09-10T10:11:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 -- 172.21.15.45:0/1366172331 wait complete. 2024-09-10T10:11:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.337+0000 7f241b1da740 1 -- start start 2024-09-10T10:11:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 1 Processor -- start 2024-09-10T10:11:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b9253c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562b92541880 con 0x562b9253ecb0 2024-09-10T10:11:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x562b92421290 con 0x562b92534410 2024-09-10T10:11:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x562b92541a00 con 0x562b92533af0 2024-09-10T10:11:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f241b1da740 10 client.0 _ll_register_callbacks cb 0x562b92452140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24159e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24169e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b9253c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24159e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:36452/0 (socket says 172.21.15.45:36452) 2024-09-10T10:11:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24159e0640 1 -- 172.21.15.45:0/3352582700 learned_addr learned my addr 172.21.15.45:0/3352582700 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24161e1640 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24169e2640 1 -- 172.21.15.45:0/3352582700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 msgr2=0x562b9253bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24169e2640 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24169e2640 1 -- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 msgr2=0x562b9253c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24169e2640 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24169e2640 1 -- 172.21.15.45:0/3352582700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b921ab210 con 0x562b9253ecb0 2024-09-10T10:11:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.338+0000 7f24159e0640 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:11:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f24169e2640 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b9253c5b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2408009ca0 tx=0x7f24080095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2c7a693b8335da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f23f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:11:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f23f9ffb640 1 -- 172.21.15.45:0/3352582700 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408004820 con 0x562b9253ecb0 2024-09-10T10:11:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f23f9ffb640 1 -- 172.21.15.45:0/3352582700 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f24080305a0 con 0x562b9253ecb0 2024-09-10T10:11:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c017070 con 0x562b9253ecb0 2024-09-10T10:11:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f23f9ffb640 1 -- 172.21.15.45:0/3352582700 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f24080307a0 con 0x562b9253ecb0 2024-09-10T10:11:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 10 client.15657 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562b92277620 con 0x562b9253ecb0 2024-09-10T10:11:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f23f9ffb640 1 -- 172.21.15.45:0/3352582700 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240c017070 con 0x562b9253ecb0 2024-09-10T10:11:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f23f9ffb640 1 -- 172.21.15.45:0/3352582700 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f2408048040 con 0x562b9253ecb0 2024-09-10T10:11:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 10 client.15657 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 20 client.15657 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b9254eb80 0x562b92550f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 4 client.15657 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:11:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.339+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b924a5a20 con 0x562b9254eb80 2024-09-10T10:11:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.340+0000 7f24161e1640 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b9254eb80 0x562b92550f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.340+0000 7f24161e1640 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b9254eb80 0x562b92550f70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f7e55e420b0fa8 server_cookie=9b97a2a89213eaff in_seq=0 out_seq=0 2024-09-10T10:11:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.340+0000 7f23f9ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f23f9ffb640 1 -- 172.21.15.45:0/3352582700 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b924a5a20 con 0x562b9254eb80 2024-09-10T10:11:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f23f9ffb640 10 client.15657 handle_command_reply: tid=0 2024-09-10T10:11:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 client.15657 shutdown 2024-09-10T10:11:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 msgr2=0x562b9253c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b9253c5b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2408009ca0 tx=0x7f24080095c0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 shutdown_connections 2024-09-10T10:11:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b9254eb80 0x562b92550f70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b9253c5b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 --2- 172.21.15.45:0/3352582700 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f23f9ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 >> 172.21.15.45:0/3352582700 conn(0x562b92530060 msgr2=0x562b92535d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 shutdown_connections 2024-09-10T10:11:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 1 -- 172.21.15.45:0/3352582700 wait complete. 2024-09-10T10:11:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.341+0000 7f241b1da740 20 client.15657 trim_cache size 0 max 16384 2024-09-10T10:11:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.356+0000 7f241b1da740 1 Processor -- start 2024-09-10T10:11:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.356+0000 7f241b1da740 1 -- start start 2024-09-10T10:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f241b1da740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f241b1da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f241b1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92556860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f241b1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24101adad0 con 0x562b9253ecb0 2024-09-10T10:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f241b1da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f24101b0330 con 0x562b92534410 2024-09-10T10:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f241b1da740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f24101ae120 con 0x562b92533af0 2024-09-10T10:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92556860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.384 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92556860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45342/0 (socket says 172.21.15.45:45342) 2024-09-10T10:11:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 -- 172.21.15.45:0/608430956 learned_addr learned my addr 172.21.15.45:0/608430956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24161e1640 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24159e0640 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 -- 172.21.15.45:0/608430956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 msgr2=0x562b9253c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 -- 172.21.15.45:0/608430956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 msgr2=0x562b9253daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 -- 172.21.15.45:0/608430956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b925512e0 con 0x562b9253ecb0 2024-09-10T10:11:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f24169e2640 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92556860 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2408009cd0 tx=0x7f2408003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4384042213ed7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f23fbfff640 1 -- 172.21.15.45:0/608430956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408004620 con 0x562b9253ecb0 2024-09-10T10:11:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.357+0000 7f23fbfff640 1 -- 172.21.15.45:0/608430956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2408036070 con 0x562b9253ecb0 2024-09-10T10:11:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f23fbfff640 1 -- 172.21.15.45:0/608430956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408030dc0 con 0x562b9253ecb0 2024-09-10T10:11:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 -- 172.21.15.45:0/608430956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 msgr2=0x562b92556860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92556860 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2408009cd0 tx=0x7f2408003f60 comp rx=0 tx=0).stop 2024-09-10T10:11:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 -- 172.21.15.45:0/608430956 shutdown_connections 2024-09-10T10:11:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92556860 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b9253daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 --2- 172.21.15.45:0/608430956 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b9253c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 -- 172.21.15.45:0/608430956 >> 172.21.15.45:0/608430956 conn(0x562b92530060 msgr2=0x562b9253d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 -- 172.21.15.45:0/608430956 shutdown_connections 2024-09-10T10:11:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 -- 172.21.15.45:0/608430956 wait complete. 2024-09-10T10:11:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.358+0000 7f241b1da740 1 -- start start 2024-09-10T10:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 1 Processor -- start 2024-09-10T10:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b925470f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b92538500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92538a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24101ae7f0 con 0x562b9253ecb0 2024-09-10T10:11:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f24101afc60 con 0x562b92534410 2024-09-10T10:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f24101aeec0 con 0x562b92533af0 2024-09-10T10:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f241b1da740 10 client.0 _ll_register_callbacks cb 0x7f24080317a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24169e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92538a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24161e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b925470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24161e1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b925470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:43088/0 (socket says 172.21.15.45:43088) 2024-09-10T10:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24159e0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b92538500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24169e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92538a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45348/0 (socket says 172.21.15.45:45348) 2024-09-10T10:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24161e1640 1 -- 172.21.15.45:0/1554188633 learned_addr learned my addr 172.21.15.45:0/1554188633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24169e2640 1 -- 172.21.15.45:0/1554188633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 msgr2=0x562b925470f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24169e2640 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b925470f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24169e2640 1 -- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 msgr2=0x562b92538500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24169e2640 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b92538500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24169e2640 1 -- 172.21.15.45:0/1554188633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b92538f80 con 0x562b9253ecb0 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24159e0640 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b92538500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.359+0000 7f24161e1640 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b925470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f24169e2640 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92538a40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2408009430 tx=0x7f2408004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9e98f4a42000fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f23f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f23f9ffb640 1 -- 172.21.15.45:0/1554188633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2408054650 con 0x562b9253ecb0 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f23f9ffb640 1 -- 172.21.15.45:0/1554188633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f24080547f0 con 0x562b9253ecb0 2024-09-10T10:11:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2408036070 con 0x562b9253ecb0 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f23f9ffb640 1 -- 172.21.15.45:0/1554188633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2408071050 con 0x562b9253ecb0 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 10 client.15663 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x562b92545980 con 0x562b9253ecb0 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f23f9ffb640 1 -- 172.21.15.45:0/1554188633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2408036070 con 0x562b9253ecb0 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f23f9ffb640 1 -- 172.21.15.45:0/1554188633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f240806c030 con 0x562b9253ecb0 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 10 client.15663 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 20 client.15663 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b925b9b70 0x562b925bbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 4 client.15663 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.360+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b9249fd70 con 0x562b925b9b70 2024-09-10T10:11:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.361+0000 7f24161e1640 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b925b9b70 0x562b925bbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.361+0000 7f24161e1640 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b925b9b70 0x562b925bbf60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e41221f138764a server_cookie=f3b5491936d2f9fe in_seq=0 out_seq=0 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.361+0000 7f23f9ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f23f9ffb640 1 -- 172.21.15.45:0/1554188633 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b9249fd70 con 0x562b925b9b70 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f23f9ffb640 10 client.15663 handle_command_reply: tid=0 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 client.15663 shutdown 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 msgr2=0x562b92538a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92538a40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2408009430 tx=0x7f2408004990 comp rx=0 tx=0).stop 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 shutdown_connections 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x562b925b9b70 0x562b925bbf60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562b9253ecb0 0x562b92538a40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x562b92534410 0x562b92538500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f23f9ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 --2- 172.21.15.45:0/1554188633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x562b92533af0 0x562b925470f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 >> 172.21.15.45:0/1554188633 conn(0x562b92530060 msgr2=0x562b92535d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 shutdown_connections 2024-09-10T10:11:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 1 -- 172.21.15.45:0/1554188633 wait complete. 2024-09-10T10:11:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.366+0000 7f241b1da740 20 client.15663 trim_cache size 0 max 16384 2024-09-10T10:11:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:11:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:11:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:11:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:11:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23fc0a2da0 msgr2=0x7f23fc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23fc0a2da0 0x7f23fc0a5260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2408004ab0 tx=0x7f2408004a40 comp rx=0 tx=0).stop 2024-09-10T10:11:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 msgr2=0x7f24100fd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f24100fd140 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f240000d950 tx=0x7f240000de10 comp rx=0 tx=0).stop 2024-09-10T10:11:38.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 shutdown_connections 2024-09-10T10:11:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23fc0a2da0 0x7f23fc0a5260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2410106a00 0x7f24100fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2410101390 0x7f24100fd140 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 --2- 172.21.15.45:0/2298173274 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f24101009f0 0x7f24100fcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 >> 172.21.15.45:0/2298173274 conn(0x7f24100fc6b0 msgr2=0x7f241006a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 shutdown_connections 2024-09-10T10:11:38.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:38.367+0000 7f2418445640 1 -- 172.21.15.45:0/2298173274 wait complete. 2024-09-10T10:11:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:39 smithi162 ceph-mon[34963]: pgmap v261: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:40.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:39 smithi045 ceph-mon[31114]: pgmap v261: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:40.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:39 smithi047 ceph-mon[35244]: pgmap v261: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:42.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:41 smithi162 ceph-mon[34963]: pgmap v262: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:11:42.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:41 smithi045 ceph-mon[31114]: pgmap v262: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:11:42.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:41 smithi047 ceph-mon[35244]: pgmap v262: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:11:44.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:43 smithi162 ceph-mon[34963]: pgmap v263: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T10:11:44.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:43 smithi045 ceph-mon[31114]: pgmap v263: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T10:11:44.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:43 smithi047 ceph-mon[35244]: pgmap v263: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T10:11:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:45 smithi162 ceph-mon[34963]: pgmap v264: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:46.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:45 smithi045 ceph-mon[31114]: pgmap v264: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:46.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:45 smithi047 ceph-mon[35244]: pgmap v264: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:48.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:47 smithi162 ceph-mon[34963]: pgmap v265: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:48.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:47 smithi045 ceph-mon[31114]: pgmap v265: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:48.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:47 smithi047 ceph-mon[35244]: pgmap v265: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:49 smithi162 ceph-mon[34963]: pgmap v266: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:49 smithi045 ceph-mon[31114]: pgmap v266: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:50.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:49 smithi047 ceph-mon[35244]: pgmap v266: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.468+0000 7f1fcf59e640 1 -- 172.21.15.45:0/3851153300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd01007f0 msgr2=0x7f1fd0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.468+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/3851153300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd01007f0 0x7f1fd0100c50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009930 tx=0x7f1fc402a420 comp rx=0 tx=0).stop 2024-09-10T10:11:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.468+0000 7f1fcf59e640 1 -- 172.21.15.45:0/3851153300 shutdown_connections 2024-09-10T10:11:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.468+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/3851153300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd0101190 0x7f1fd010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.468+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/3851153300 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd01007f0 0x7f1fd0100c50 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.468+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/3851153300 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd0106800 0x7f1fd0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.468+0000 7f1fcf59e640 1 -- 172.21.15.45:0/3851153300 >> 172.21.15.45:0/3851153300 conn(0x7f1fd00fc510 msgr2=0x7f1fd00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.469+0000 7f1fcf59e640 1 -- 172.21.15.45:0/3851153300 shutdown_connections 2024-09-10T10:11:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.469+0000 7f1fcf59e640 1 -- 172.21.15.45:0/3851153300 wait complete. 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.469+0000 7f1fcf59e640 1 Processor -- start 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.469+0000 7f1fcf59e640 1 -- start start 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcf59e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd01007f0 0x7f1fd00fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcf59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd0101190 0x7f1fd00fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcf59e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd0106800 0x7f1fd00fd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcf59e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0110c40 con 0x7f1fd0101190 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcf59e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0110ac0 con 0x7f1fd01007f0 2024-09-10T10:11:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcf59e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd0110dc0 con 0x7f1fd0106800 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd0101190 0x7f1fd00fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd0101190 0x7f1fd00fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38876/0 (socket says 172.21.15.45:38876) 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 -- 172.21.15.45:0/774575896 learned_addr learned my addr 172.21.15.45:0/774575896 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fce59c640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd01007f0 0x7f1fd00fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fced9d640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd0106800 0x7f1fd00fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 -- 172.21.15.45:0/774575896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd0106800 msgr2=0x7f1fd00fd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd0106800 0x7f1fd00fd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 -- 172.21.15.45:0/774575896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd01007f0 msgr2=0x7f1fd00fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd01007f0 0x7f1fd00fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 -- 172.21.15.45:0/774575896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd01a6790 con 0x7f1fd0101190 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.470+0000 7f1fcdd9b640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd0101190 0x7f1fd00fd110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009230 tx=0x7f1fc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.471+0000 7f1fbf7fe640 1 -- 172.21.15.45:0/774575896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc403f030 con 0x7f1fd0101190 2024-09-10T10:11:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.471+0000 7f1fbf7fe640 1 -- 172.21.15.45:0/774575896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc4003ad0 con 0x7f1fd0101190 2024-09-10T10:11:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.471+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd01a6a20 con 0x7f1fd0101190 2024-09-10T10:11:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.471+0000 7f1fbf7fe640 1 -- 172.21.15.45:0/774575896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4035070 con 0x7f1fd0101190 2024-09-10T10:11:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.471+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd01a6f10 con 0x7f1fd0101190 2024-09-10T10:11:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.473+0000 7f1fbf7fe640 1 -- 172.21.15.45:0/774575896 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1fc4042030 con 0x7f1fd0101190 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 Processor -- start 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 -- start start 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e55fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e56500 0x5609a8e69150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e69690 0x5609a8e6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5609a8dba2f0 con 0x5609a8e56500 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609a8d62270 con 0x5609a8e55be0 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5609a8db62b0 con 0x5609a8e69690 2024-09-10T10:11:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fbf7fe640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f1fb80a2cf0 0x7f1fb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fce59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e56500 0x5609a8e69150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fce59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e56500 0x5609a8e69150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38886/0 (socket says 172.21.15.45:38886) 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fce59c640 1 -- 172.21.15.45:0/1210285312 learned_addr learned my addr 172.21.15.45:0/1210285312 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fced9d640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f1fb80a2cf0 0x7f1fb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.474+0000 7f1fbf7fe640 1 -- 172.21.15.45:0/774575896 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f1fc40ef8c0 con 0x7f1fd0101190 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e55fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fcdd9b640 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e69690 0x5609a8e6ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fce59c640 1 -- 172.21.15.45:0/1210285312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e69690 msgr2=0x5609a8e6ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e69690 0x5609a8e6ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fce59c640 1 -- 172.21.15.45:0/1210285312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 msgr2=0x5609a8e55fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e55fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fce59c640 1 -- 172.21.15.45:0/1210285312 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a8d61340 con 0x5609a8e56500 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e55fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fced9d640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f1fb80a2cf0 0x7f1fb80a51b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8005fd0 tx=0x7f1fc8009b50 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e56500 0x5609a8e69150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000bc90 tx=0x7f1fc00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d2322c40720afb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fbcff9640 1 -- 172.21.15.45:0/1210285312 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc001c480 con 0x5609a8e56500 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fbcff9640 1 -- 172.21.15.45:0/1210285312 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc001cb00 con 0x5609a8e56500 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.475+0000 7f1fbcff9640 1 -- 172.21.15.45:0/1210285312 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0011bb0 con 0x5609a8e56500 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1210285312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e56500 msgr2=0x5609a8e69150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e56500 0x5609a8e69150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000bc90 tx=0x7f1fc00107a0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1210285312 shutdown_connections 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e69690 0x5609a8e6ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e56500 0x5609a8e69150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1210285312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e55fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1210285312 >> 172.21.15.45:0/1210285312 conn(0x5609a8e52230 msgr2=0x5609a8e52610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1210285312 shutdown_connections 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1210285312 wait complete. 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.476+0000 7f1fd7a6e740 1 -- start start 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 1 Processor -- start 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e55be0 0x5609a8e66530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e56500 0x5609a8e66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e66fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5609a8e6c260 con 0x5609a8e69690 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609a8d43290 con 0x5609a8e56500 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5609a8e6c3e0 con 0x5609a8e55be0 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fd7a6e740 10 client.0 _ll_register_callbacks cb 0x5609a8d74140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e66fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fcdd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e56500 0x5609a8e66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e66fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38896/0 (socket says 172.21.15.45:38896) 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fcdd9b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e56500 0x5609a8e66a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:36034/0 (socket says 172.21.15.45:36034) 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 -- 172.21.15.45:0/1214666344 learned_addr learned my addr 172.21.15.45:0/1214666344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e55be0 0x5609a8e66530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 -- 172.21.15.45:0/1214666344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e55be0 msgr2=0x5609a8e66530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e55be0 0x5609a8e66530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 -- 172.21.15.45:0/1214666344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e56500 msgr2=0x5609a8e66a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e56500 0x5609a8e66a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.477+0000 7f1fced9d640 1 -- 172.21.15.45:0/1214666344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a8acd210 con 0x5609a8e69690 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e66fb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8010b70 tx=0x7f1fc8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49e14cdda5ad821 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1214666344 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8025070 con 0x5609a8e69690 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1214666344 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f1fc8012230 con 0x5609a8e69690 2024-09-10T10:11:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc001cb00 con 0x5609a8e69690 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1214666344 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1fc801eb10 con 0x5609a8e69690 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 10 client.15675 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5609a8b99620 con 0x5609a8e69690 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1214666344 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc001cb00 con 0x5609a8e69690 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1214666344 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1fc801f3f0 con 0x5609a8e69690 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 10 client.15675 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 20 client.15675 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8e79630 0x5609a8e7ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 4 client.15675 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.478+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a8dc7a20 con 0x5609a8e79630 2024-09-10T10:11:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.479+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8e79630 0x5609a8e7ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.479+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8e79630 0x5609a8e7ba80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53f1157ea1e76aa server_cookie=ff1a7c5f4b50a9ea in_seq=0 out_seq=0 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.479+0000 7f1fb5ffb640 10 client.15675 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1214666344 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609a8dc7a20 con 0x5609a8e79630 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fb5ffb640 10 client.15675 handle_command_reply: tid=0 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 client.15675 shutdown 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 msgr2=0x5609a8e66fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e66fb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8010b70 tx=0x7f1fc8004ab0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 shutdown_connections 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8e79630 0x5609a8e7ba80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e66fb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e56500 0x5609a8e66a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1214666344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e55be0 0x5609a8e66530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fb5ffb640 0 client.15675 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 >> 172.21.15.45:0/1214666344 conn(0x5609a8e52230 msgr2=0x5609a8e57ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 shutdown_connections 2024-09-10T10:11:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1214666344 wait complete. 2024-09-10T10:11:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.480+0000 7f1fd7a6e740 20 client.15675 trim_cache size 0 max 16384 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 Processor -- start 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 -- start start 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e66c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e813e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e81920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01a9560 con 0x5609a8e69690 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01abdc0 con 0x5609a8e55be0 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01a9bb0 con 0x5609a8e56500 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fced9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e81920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fced9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e81920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38904/0 (socket says 172.21.15.45:38904) 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fced9d640 1 -- 172.21.15.45:0/4189240447 learned_addr learned my addr 172.21.15.45:0/4189240447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.495+0000 7f1fce59c640 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fced9d640 1 -- 172.21.15.45:0/4189240447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 msgr2=0x5609a8e813e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fcdd9b640 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fced9d640 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e813e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fced9d640 1 -- 172.21.15.45:0/4189240447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 msgr2=0x5609a8e66c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fced9d640 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e66c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fced9d640 1 -- 172.21.15.45:0/4189240447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a8e52060 con 0x5609a8e69690 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fcdd9b640 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e813e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fced9d640 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e81920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1fc801a040 tx=0x7f1fc801e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bbea318f2cd815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fb7fff640 1 -- 172.21.15.45:0/4189240447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8035070 con 0x5609a8e69690 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fb7fff640 1 -- 172.21.15.45:0/4189240447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc801fad0 con 0x5609a8e69690 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.496+0000 7f1fb7fff640 1 -- 172.21.15.45:0/4189240447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8025040 con 0x5609a8e69690 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/4189240447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 msgr2=0x5609a8e81920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e81920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1fc801a040 tx=0x7f1fc801e890 comp rx=0 tx=0).stop 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/4189240447 shutdown_connections 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e81920 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e813e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/4189240447 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e66c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/4189240447 >> 172.21.15.45:0/4189240447 conn(0x5609a8e52230 msgr2=0x5609a8e664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/4189240447 shutdown_connections 2024-09-10T10:11:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/4189240447 wait complete. 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- start start 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 Processor -- start 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e719c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e71f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e6c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01aa280 con 0x5609a8e69690 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01ab6f0 con 0x5609a8e55be0 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd01aa950 con 0x5609a8e56500 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fd7a6e740 10 client.0 _ll_register_callbacks cb 0x7f1fc8012530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fced9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e6c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fced9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e6c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38914/0 (socket says 172.21.15.45:38914) 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.497+0000 7f1fced9d640 1 -- 172.21.15.45:0/1444058117 learned_addr learned my addr 172.21.15.45:0/1444058117 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fcdd9b640 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e71f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fced9d640 1 -- 172.21.15.45:0/1444058117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 msgr2=0x5609a8e71f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e71f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fced9d640 1 -- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 msgr2=0x5609a8e719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fced9d640 1 -- 172.21.15.45:0/1444058117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a8e6cfd0 con 0x5609a8e69690 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fced9d640 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e6c7a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009ea0 tx=0x7f1fc801fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50023e2546748002 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:11:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1444058117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8025070 con 0x5609a8e69690 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1444058117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f1fc8004ba0 con 0x5609a8e69690 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc801fad0 con 0x5609a8e69690 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1444058117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1fc8035070 con 0x5609a8e69690 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fd7a6e740 10 client.15681 fetch_fsmap learned FSMap version 21 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5609a8ee4f90 con 0x5609a8e69690 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.498+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1444058117 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc801fad0 con 0x5609a8e69690 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1444058117 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f1fc803a460 con 0x5609a8e69690 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fd7a6e740 10 client.15681 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fd7a6e740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fd7a6e740 20 client.15681 populate_metadata read hostname 'smithi045' 2024-09-10T10:11:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8ee55f0 0x5609a8ee79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:11:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fd7a6e740 4 client.15681 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:11:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609a8dc1d70 con 0x5609a8ee55f0 2024-09-10T10:11:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8ee55f0 0x5609a8ee79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:11:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fb5ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.499+0000 7f1fce59c640 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8ee55f0 0x5609a8ee79e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb251f0497d268e server_cookie=55fc3b416d3d043b in_seq=0 out_seq=0 2024-09-10T10:11:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fb5ffb640 1 -- 172.21.15.45:0/1444058117 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609a8dc1d70 con 0x5609a8ee55f0 2024-09-10T10:11:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fb5ffb640 10 client.15681 handle_command_reply: tid=0 2024-09-10T10:11:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 client.15681 shutdown 2024-09-10T10:11:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 msgr2=0x5609a8e6c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e6c7a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009ea0 tx=0x7f1fc801fd10 comp rx=0 tx=0).stop 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 shutdown_connections 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5609a8ee55f0 0x5609a8ee79e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5609a8e69690 0x5609a8e6c7a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5609a8e56500 0x5609a8e71f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fb5ffb640 0 client.15681 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 --2- 172.21.15.45:0/1444058117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8e55be0 0x5609a8e719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 >> 172.21.15.45:0/1444058117 conn(0x5609a8e52230 msgr2=0x5609a8e66d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 shutdown_connections 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 1 -- 172.21.15.45:0/1444058117 wait complete. 2024-09-10T10:11:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.504+0000 7f1fd7a6e740 20 client.15681 trim_cache size 0 max 16384 2024-09-10T10:11:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:11:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:11:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:11:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f1fb80a2cf0 msgr2=0x7f1fb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f1fb80a2cf0 0x7f1fb80a51b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8005fd0 tx=0x7f1fc8009b50 comp rx=0 tx=0).stop 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd0101190 msgr2=0x7f1fd00fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd0101190 0x7f1fd00fd110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009230 tx=0x7f1fc4004270 comp rx=0 tx=0).stop 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 shutdown_connections 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f1fd0106800 0x7f1fd00fd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1fd0101190 0x7f1fd00fd110 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f1fb80a2cf0 0x7f1fb80a51b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 --2- 172.21.15.45:0/774575896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1fd01007f0 0x7f1fd00fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 >> 172.21.15.45:0/774575896 conn(0x7f1fd00fc510 msgr2=0x7f1fd006a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:11:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 shutdown_connections 2024-09-10T10:11:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:11:51.505+0000 7f1fcf59e640 1 -- 172.21.15.45:0/774575896 wait complete. 2024-09-10T10:11:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:51 smithi162 ceph-mon[34963]: pgmap v267: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:52.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:51 smithi045 ceph-mon[31114]: pgmap v267: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:52.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:51 smithi047 ceph-mon[35244]: pgmap v267: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:11:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:53 smithi162 ceph-mon[34963]: pgmap v268: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 767 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:11:54.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:53 smithi045 ceph-mon[31114]: pgmap v268: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 767 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:11:54.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:53 smithi047 ceph-mon[35244]: pgmap v268: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 767 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:11:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:55 smithi162 ceph-mon[34963]: pgmap v269: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-10T10:11:56.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:55 smithi045 ceph-mon[31114]: pgmap v269: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-10T10:11:56.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:55 smithi047 ceph-mon[35244]: pgmap v269: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-10T10:11:58.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:57 smithi162 ceph-mon[34963]: pgmap v270: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T10:11:58.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:57 smithi045 ceph-mon[31114]: pgmap v270: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T10:11:58.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:57 smithi047 ceph-mon[35244]: pgmap v270: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T10:12:00.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:11:59 smithi162 ceph-mon[34963]: pgmap v271: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:00.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:11:59 smithi045 ceph-mon[31114]: pgmap v271: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:00.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:11:59 smithi047 ceph-mon[35244]: pgmap v271: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:02.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:01 smithi162 ceph-mon[34963]: pgmap v272: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:02.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:01 smithi045 ceph-mon[31114]: pgmap v272: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:02.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:01 smithi047 ceph-mon[35244]: pgmap v272: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:04.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:03 smithi162 ceph-mon[34963]: pgmap v273: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:04.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:03 smithi045 ceph-mon[31114]: pgmap v273: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:04.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:03 smithi047 ceph-mon[35244]: pgmap v273: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:04.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 -- 172.21.15.45:0/1192152925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 msgr2=0x7f636410ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 --2- 172.21.15.45:0/1192152925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f636410ba80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6358009ca0 tx=0x7f635802a950 comp rx=0 tx=0).stop 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 -- 172.21.15.45:0/1192152925 shutdown_connections 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 --2- 172.21.15.45:0/1192152925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f636410c0d0 0x7f636410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 --2- 172.21.15.45:0/1192152925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f636410ba80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 --2- 172.21.15.45:0/1192152925 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6364102c80 0x7f6364103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 -- 172.21.15.45:0/1192152925 >> 172.21.15.45:0/1192152925 conn(0x7f6364077990 msgr2=0x7f63640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 -- 172.21.15.45:0/1192152925 shutdown_connections 2024-09-10T10:12:04.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.605+0000 7f636914b640 1 -- 172.21.15.45:0/1192152925 wait complete. 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 Processor -- start 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 -- start start 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6364102c80 0x7f6364198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f6364198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f636410c0d0 0x7f636419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6364111eb0 con 0x7f63641035a0 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6364111d30 con 0x7f636410c0d0 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636914b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f6364112030 con 0x7f6364102c80 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f6364198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f6364198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55372/0 (socket says 172.21.15.45:55372) 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636259c640 1 -- 172.21.15.45:0/490664674 learned_addr learned my addr 172.21.15.45:0/490664674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f636359e640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f636410c0d0 0x7f636419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.606+0000 7f6362d9d640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6364102c80 0x7f6364198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636259c640 1 -- 172.21.15.45:0/490664674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6364102c80 msgr2=0x7f6364198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636259c640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6364102c80 0x7f6364198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636259c640 1 -- 172.21.15.45:0/490664674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f636410c0d0 msgr2=0x7f636419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636259c640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f636410c0d0 0x7f636419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636259c640 1 -- 172.21.15.45:0/490664674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636419d2f0 con 0x7f63641035a0 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f6362d9d640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6364102c80 0x7f6364198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636259c640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f6364198840 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6358005cc0 tx=0x7f635802f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f634bfff640 1 -- 172.21.15.45:0/490664674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358034040 con 0x7f63641035a0 2024-09-10T10:12:04.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636419d580 con 0x7f63641035a0 2024-09-10T10:12:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f634bfff640 1 -- 172.21.15.45:0/490664674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6358003e40 con 0x7f63641035a0 2024-09-10T10:12:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f634bfff640 1 -- 172.21.15.45:0/490664674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358045480 con 0x7f63641035a0 2024-09-10T10:12:04.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.607+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63641a4e70 con 0x7f63641035a0 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f634bfff640 1 -- 172.21.15.45:0/490664674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6358037020 con 0x7f63641035a0 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 Processor -- start 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 -- start start 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2bccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bdbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bde4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c6b2b312f0 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c6b2ad9270 con 0x55c6b2bcd330 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636bfc5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c6b2b2d2b0 con 0x55c6b2bcca10 2024-09-10T10:12:04.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bde4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f6362d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bdbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bde4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55388/0 (socket says 172.21.15.45:55388) 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f6362d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bdbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:41588/0 (socket says 172.21.15.45:41588) 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636259c640 1 -- 172.21.15.45:0/3143910414 learned_addr learned my addr 172.21.15.45:0/3143910414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.610+0000 7f636359e640 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2bccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f634bfff640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f634c0a2da0 0x7f634c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f6362d9d640 1 -- 172.21.15.45:0/3143910414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 msgr2=0x55c6b2bccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f6362d9d640 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2bccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f6362d9d640 1 -- 172.21.15.45:0/3143910414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 msgr2=0x55c6b2bde4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f6362d9d640 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bde4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f6362d9d640 1 -- 172.21.15.45:0/3143910414 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b2ad8340 con 0x55c6b2bcd330 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f636259c640 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bde4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:12:04.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f636359e640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f634c0a2da0 0x7f634c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f634bfff640 1 -- 172.21.15.45:0/490664674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f63580f0450 con 0x7f63641035a0 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f6362d9d640 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bdbb80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6354008b80 tx=0x7f635400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=654c5a45da4558bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f63497fa640 1 -- 172.21.15.45:0/3143910414 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354014070 con 0x55c6b2bcd330 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f636359e640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f634c0a2da0 0x7f634c0a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f635c005fd0 tx=0x7f635c0046c0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f63497fa640 1 -- 172.21.15.45:0/3143910414 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63540047a0 con 0x55c6b2bcd330 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.611+0000 7f63497fa640 1 -- 172.21.15.45:0/3143910414 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354010650 con 0x55c6b2bcd330 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 -- 172.21.15.45:0/3143910414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 msgr2=0x55c6b2bdbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bdbb80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6354008b80 tx=0x7f635400e720 comp rx=0 tx=0).stop 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 -- 172.21.15.45:0/3143910414 shutdown_connections 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bde4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bdbb80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 --2- 172.21.15.45:0/3143910414 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2bccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 -- 172.21.15.45:0/3143910414 >> 172.21.15.45:0/3143910414 conn(0x55c6b2bc9060 msgr2=0x55c6b2bc9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:04.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 -- 172.21.15.45:0/3143910414 shutdown_connections 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 -- 172.21.15.45:0/3143910414 wait complete. 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 -- start start 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.612+0000 7f636bfc5740 1 Processor -- start 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcca10 0x55c6b2bd8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bd94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bd99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636bfc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c6b2bdec90 con 0x55c6b2bcca10 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636bfc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c6b2aba290 con 0x55c6b2bcd330 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636bfc5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c6b2bdee10 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636bfc5740 10 client.0 _ll_register_callbacks cb 0x55c6b2aeb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcca10 0x55c6b2bd8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcca10 0x55c6b2bd8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55404/0 (socket says 172.21.15.45:55404) 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 -- 172.21.15.45:0/1310946110 learned_addr learned my addr 172.21.15.45:0/1310946110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:04.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636259c640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bd94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636359e640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bd99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 -- 172.21.15.45:0/1310946110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 msgr2=0x55c6b2bd99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bd99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 -- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 msgr2=0x55c6b2bd94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bd94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 -- 172.21.15.45:0/1310946110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6b2844210 con 0x55c6b2bcca10 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f636359e640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bd99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6362d9d640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcca10 0x55c6b2bd8f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6354005160 tx=0x7f6354005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc65f8b2a3c04b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6346ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.613+0000 7f6346ff5640 1 -- 172.21.15.45:0/1310946110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354014070 con 0x55c6b2bcca10 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f6346ff5640 1 -- 172.21.15.45:0/1310946110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f6354004030 con 0x55c6b2bcca10 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63540047a0 con 0x55c6b2bcca10 2024-09-10T10:12:04.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f6346ff5640 1 -- 172.21.15.45:0/1310946110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6354004d60 con 0x55c6b2bcca10 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 10 client.15693 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c6b2910620 con 0x55c6b2bcca10 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f6346ff5640 1 -- 172.21.15.45:0/1310946110 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63540047a0 con 0x55c6b2bcca10 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f6346ff5640 1 -- 172.21.15.45:0/1310946110 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f6354003c70 con 0x55c6b2bcca10 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 10 client.15693 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 20 client.15693 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2bebfa0 0x55c6b2bee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 4 client.15693 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6b2b3ea20 con 0x55c6b2bebfa0 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.614+0000 7f636259c640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2bebfa0 0x55c6b2bee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.615+0000 7f6346ff5640 10 client.15693 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.615+0000 7f636259c640 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2bebfa0 0x55c6b2bee390 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0023270207539d1 server_cookie=88d8a55f6b420405 in_seq=0 out_seq=0 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f6346ff5640 1 -- 172.21.15.45:0/1310946110 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6b2b3ea20 con 0x55c6b2bebfa0 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f6346ff5640 10 client.15693 handle_command_reply: tid=0 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 client.15693 shutdown 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcca10 msgr2=0x55c6b2bd8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcca10 0x55c6b2bd8f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6354005160 tx=0x7f6354005190 comp rx=0 tx=0).stop 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 shutdown_connections 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2bebfa0 0x55c6b2bee390 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bd99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2bd94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f6346ff5640 0 client.15693 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 --2- 172.21.15.45:0/1310946110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcca10 0x55c6b2bd8f60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 >> 172.21.15.45:0/1310946110 conn(0x55c6b2bc9060 msgr2=0x55c6b2bc81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 shutdown_connections 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 1 -- 172.21.15.45:0/1310946110 wait complete. 2024-09-10T10:12:04.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.616+0000 7f636bfc5740 20 client.15693 trim_cache size 0 max 16384 2024-09-10T10:12:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.630+0000 7f636bfc5740 1 Processor -- start 2024-09-10T10:12:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636bfc5740 1 -- start start 2024-09-10T10:12:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcca10 0x55c6b2bc80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcd330 0x55c6b2bf3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bf3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636bfc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63641a95a0 con 0x55c6b2bcd330 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636bfc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f63641abe00 con 0x55c6b2bcca10 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636bfc5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63641a9bf0 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcd330 0x55c6b2bf3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcd330 0x55c6b2bf3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55414/0 (socket says 172.21.15.45:55414) 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 -- 172.21.15.45:0/581586352 learned_addr learned my addr 172.21.15.45:0/581586352 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f6362d9d640 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcca10 0x55c6b2bc80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 -- 172.21.15.45:0/581586352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 msgr2=0x55c6b2bf3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bf3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 -- 172.21.15.45:0/581586352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcca10 msgr2=0x55c6b2bc80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcca10 0x55c6b2bc80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 -- 172.21.15.45:0/581586352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6b2bcd160 con 0x55c6b2bcd330 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.631+0000 7f636259c640 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcd330 0x55c6b2bf3930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f63580bc760 tx=0x7f63580f6060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2df1559dc6bfeaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:04.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f6348ff9640 1 -- 172.21.15.45:0/581586352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63580bc3c0 con 0x55c6b2bcd330 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f6348ff9640 1 -- 172.21.15.45:0/581586352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63580bd160 con 0x55c6b2bcd330 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f6348ff9640 1 -- 172.21.15.45:0/581586352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63580badf0 con 0x55c6b2bcd330 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 -- 172.21.15.45:0/581586352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcd330 msgr2=0x55c6b2bf3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcd330 0x55c6b2bf3930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f63580bc760 tx=0x7f63580f6060 comp rx=0 tx=0).stop 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 -- 172.21.15.45:0/581586352 shutdown_connections 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2bf3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bcd330 0x55c6b2bf3930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 --2- 172.21.15.45:0/581586352 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcca10 0x55c6b2bc80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 -- 172.21.15.45:0/581586352 >> 172.21.15.45:0/581586352 conn(0x55c6b2bc9060 msgr2=0x55c6b2bc79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 -- 172.21.15.45:0/581586352 shutdown_connections 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.632+0000 7f636bfc5740 1 -- 172.21.15.45:0/581586352 wait complete. 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 -- start start 2024-09-10T10:12:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 Processor -- start 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2be01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2be06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2c55900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63641aa2c0 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f63641ab730 con 0x55c6b2bcd330 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f63641aa990 con 0x55c6b2bcca10 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636bfc5740 10 client.0 _ll_register_callbacks cb 0x7f63580f3e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2c55900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2c55900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55430/0 (socket says 172.21.15.45:55430) 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 -- 172.21.15.45:0/2954901782 learned_addr learned my addr 172.21.15.45:0/2954901782 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636259c640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2be06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f6362d9d640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2be01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 -- 172.21.15.45:0/2954901782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 msgr2=0x55c6b2be01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2be01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 -- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 msgr2=0x55c6b2be06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2be06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636359e640 1 -- 172.21.15.45:0/2954901782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6b2be0ff0 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.633+0000 7f636259c640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2be06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636359e640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2c55900 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f635c020a60 tx=0x7f635c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1814a7ef88a42b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f6346ff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f6346ff5640 1 -- 172.21.15.45:0/2954901782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c029440 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f6346ff5640 1 -- 172.21.15.45:0/2954901782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8556+0+0 (secure 0 0 0) 0x7f635c00f420 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63580bd160 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f6346ff5640 1 -- 172.21.15.45:0/2954901782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f635c00f620 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 10 client.15699 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c6b2c56f80 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f6346ff5640 1 -- 172.21.15.45:0/2954901782 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63580bd160 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f6346ff5640 1 -- 172.21.15.45:0/2954901782 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f635c027070 con 0x55c6b2bdc0c0 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 10 client.15699 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 20 client.15699 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2c57720 0x55c6b2c59b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 4 client.15699 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.634+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6b2b38d70 con 0x55c6b2c57720 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.635+0000 7f6362d9d640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2c57720 0x55c6b2c59b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:04.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.635+0000 7f6362d9d640 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2c57720 0x55c6b2c59b40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeb699d3f183ba47 server_cookie=cfee8f772e62c32c in_seq=0 out_seq=0 2024-09-10T10:12:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.635+0000 7f6346ff5640 10 client.15699 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f6346ff5640 1 -- 172.21.15.45:0/2954901782 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6b2b38d70 con 0x55c6b2c57720 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f6346ff5640 10 client.15699 handle_command_reply: tid=0 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 client.15699 shutdown 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 msgr2=0x55c6b2c55900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2c55900 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f635c020a60 tx=0x7f635c005f60 comp rx=0 tx=0).stop 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 shutdown_connections 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6b2c57720 0x55c6b2c59b40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c6b2bdc0c0 0x55c6b2c55900 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c6b2bcd330 0x55c6b2be06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.646 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f6346ff5640 0 client.15699 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 --2- 172.21.15.45:0/2954901782 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c6b2bcca10 0x55c6b2be01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 >> 172.21.15.45:0/2954901782 conn(0x55c6b2bc9060 msgr2=0x55c6b2bc81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 shutdown_connections 2024-09-10T10:12:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 1 -- 172.21.15.45:0/2954901782 wait complete. 2024-09-10T10:12:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636bfc5740 20 client.15699 trim_cache size 0 max 16384 2024-09-10T10:12:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:12:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:12:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:12:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:12:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f634c0a2da0 msgr2=0x7f634c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f634c0a2da0 0x7f634c0a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f635c005fd0 tx=0x7f635c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 msgr2=0x7f6364198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f6364198840 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6358005cc0 tx=0x7f635802f6c0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 shutdown_connections 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f634c0a2da0 0x7f634c0a5260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f636410c0d0 0x7f636419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63641035a0 0x7f6364198840 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 --2- 172.21.15.45:0/490664674 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6364102c80 0x7f6364198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 >> 172.21.15.45:0/490664674 conn(0x7f6364077990 msgr2=0x7f63640fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 shutdown_connections 2024-09-10T10:12:04.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:04.644+0000 7f636914b640 1 -- 172.21.15.45:0/490664674 wait complete. 2024-09-10T10:12:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:05 smithi162 ceph-mon[34963]: pgmap v274: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:12:06.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:05 smithi045 ceph-mon[31114]: pgmap v274: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:12:06.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:05 smithi047 ceph-mon[35244]: pgmap v274: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:12:08.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:07 smithi162 ceph-mon[34963]: pgmap v275: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-09-10T10:12:08.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:07 smithi045 ceph-mon[31114]: pgmap v275: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-09-10T10:12:08.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:07 smithi047 ceph-mon[35244]: pgmap v275: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-09-10T10:12:10.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:09 smithi162 ceph-mon[34963]: pgmap v276: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:12:10.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:09 smithi045 ceph-mon[31114]: pgmap v276: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:12:10.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:09 smithi047 ceph-mon[35244]: pgmap v276: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:12:11.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:10 smithi162 ceph-mon[34963]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:12:11.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:10 smithi045 ceph-mon[31114]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:12:11.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:10 smithi047 ceph-mon[35244]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T10:12:12.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:11 smithi162 ceph-mon[34963]: pgmap v278: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 14 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-09-10T10:12:12.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:11 smithi045 ceph-mon[31114]: pgmap v278: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 14 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-09-10T10:12:12.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:11 smithi047 ceph-mon[35244]: pgmap v278: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 14 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-09-10T10:12:14.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:13 smithi162 ceph-mon[34963]: pgmap v279: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 50 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 14 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-09-10T10:12:14.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:13 smithi045 ceph-mon[31114]: pgmap v279: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 50 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 14 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-09-10T10:12:14.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:13 smithi047 ceph-mon[35244]: pgmap v279: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 50 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 14 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-09-10T10:12:16.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:15 smithi162 ceph-mon[34963]: pgmap v280: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 42 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:12:16.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:15 smithi045 ceph-mon[31114]: pgmap v280: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 42 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:12:16.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:15 smithi047 ceph-mon[35244]: pgmap v280: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 42 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:12:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 -- 172.21.15.45:0/3672563266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09841009f0 msgr2=0x7f0984100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 --2- 172.21.15.45:0/3672563266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09841009f0 0x7f0984100e50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0974009930 tx=0x7f097402a420 comp rx=0 tx=0).stop 2024-09-10T10:12:17.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 -- 172.21.15.45:0/3672563266 shutdown_connections 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 --2- 172.21.15.45:0/3672563266 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f0984101390 0x7f098410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 --2- 172.21.15.45:0/3672563266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09841009f0 0x7f0984100e50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 --2- 172.21.15.45:0/3672563266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0984106a00 0x7f0984106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 -- 172.21.15.45:0/3672563266 >> 172.21.15.45:0/3672563266 conn(0x7f09840fc6b0 msgr2=0x7f09840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 -- 172.21.15.45:0/3672563266 shutdown_connections 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.745+0000 7f098b73a640 1 -- 172.21.15.45:0/3672563266 wait complete. 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 Processor -- start 2024-09-10T10:12:17.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 -- start start 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09841009f0 0x7f0984198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0984101390 0x7f0984198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984106a00 0x7f098419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0984110e40 con 0x7f0984106a00 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0984110cc0 con 0x7f0984101390 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f098b73a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f0984110fc0 con 0x7f09841009f0 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f0989cd7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984106a00 0x7f098419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f0989cd7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984106a00 0x7f098419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41864/0 (socket says 172.21.15.45:41864) 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.746+0000 7f0989cd7640 1 -- 172.21.15.45:0/706757081 learned_addr learned my addr 172.21.15.45:0/706757081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f09894d6640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09841009f0 0x7f0984198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.750 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f0988cd5640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0984101390 0x7f0984198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f0989cd7640 1 -- 172.21.15.45:0/706757081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09841009f0 msgr2=0x7f0984198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f0989cd7640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09841009f0 0x7f0984198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f0989cd7640 1 -- 172.21.15.45:0/706757081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0984101390 msgr2=0x7f0984198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f0989cd7640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0984101390 0x7f0984198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f0989cd7640 1 -- 172.21.15.45:0/706757081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098419d410 con 0x7f0984106a00 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f0989cd7640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984106a00 0x7f098419cce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f097c00eaa0 tx=0x7f097c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f097a7fc640 1 -- 172.21.15.45:0/706757081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c00cdb0 con 0x7f0984106a00 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f097a7fc640 1 -- 172.21.15.45:0/706757081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097c005020 con 0x7f0984106a00 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098419d700 con 0x7f0984106a00 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f097a7fc640 1 -- 172.21.15.45:0/706757081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0116b0 con 0x7f0984106a00 2024-09-10T10:12:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.747+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09841a4f90 con 0x7f0984106a00 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.750+0000 7f097a7fc640 1 -- 172.21.15.45:0/706757081 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f097c005560 con 0x7f0984106a00 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.750+0000 7f098e4d4740 1 Processor -- start 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.750+0000 7f098e4d4740 1 -- start start 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.750+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726608af0 0x55d726608ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.750+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726609410 0x55d726613770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d7266160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f098e4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d72656d2f0 con 0x55d726609410 2024-09-10T10:12:17.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f098e4d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d726515270 con 0x55d726608af0 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f098e4d4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d7265692b0 con 0x55d726613cb0 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f097a7fc640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f09700a2da0 0x7f09700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726609410 0x55d726613770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726609410 0x55d726613770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41874/0 (socket says 172.21.15.45:41874) 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 -- 172.21.15.45:0/2732559585 learned_addr learned my addr 172.21.15.45:0/2732559585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f0989cd7640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f09700a2da0 0x7f09700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f0988cd5640 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726608af0 0x55d726608ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f097a7fc640 1 -- 172.21.15.45:0/706757081 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f097c0cf6f0 con 0x7f0984106a00 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f0988cd5640 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d7266160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 -- 172.21.15.45:0/2732559585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 msgr2=0x55d7266160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d7266160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 -- 172.21.15.45:0/2732559585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726608af0 msgr2=0x55d726608ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726608af0 0x55d726608ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 -- 172.21.15.45:0/2732559585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d726514340 con 0x55d726609410 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f0988cd5640 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d7266160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f0989cd7640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f09700a2da0 0x7f09700a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f097c0ccea0 tx=0x7f097c06db80 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.751+0000 7f09894d6640 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726609410 0x55d726613770 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f098000bc90 tx=0x7f09800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a00e6109a188dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f096f7fe640 1 -- 172.21.15.45:0/2732559585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098001c480 con 0x55d726609410 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f096f7fe640 1 -- 172.21.15.45:0/2732559585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098001cb00 con 0x55d726609410 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f096f7fe640 1 -- 172.21.15.45:0/2732559585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980011bb0 con 0x55d726609410 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 -- 172.21.15.45:0/2732559585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726609410 msgr2=0x55d726613770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726609410 0x55d726613770 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f098000bc90 tx=0x7f09800107a0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 -- 172.21.15.45:0/2732559585 shutdown_connections 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d7266160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726609410 0x55d726613770 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2732559585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726608af0 0x55d726608ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 -- 172.21.15.45:0/2732559585 >> 172.21.15.45:0/2732559585 conn(0x55d726605060 msgr2=0x55d726605440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.752+0000 7f098e4d4740 1 -- 172.21.15.45:0/2732559585 shutdown_connections 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 -- 172.21.15.45:0/2732559585 wait complete. 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 -- start start 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 Processor -- start 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d726610b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d726611080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d7266115c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d726616880 con 0x55d726613cb0 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d7264f6290 con 0x55d726609410 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d726616a00 con 0x55d726608af0 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.753+0000 7f098e4d4740 10 client.0 _ll_register_callbacks cb 0x55d726527140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d7266115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d7266115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41890/0 (socket says 172.21.15.45:41890) 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 -- 172.21.15.45:0/659216175 learned_addr learned my addr 172.21.15.45:0/659216175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d726610b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f09894d6640 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d726611080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 -- 172.21.15.45:0/659216175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 msgr2=0x55d726610b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d726610b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 -- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 msgr2=0x55d726611080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d726611080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 -- 172.21.15.45:0/659216175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d726280210 con 0x55d726613cb0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f0988cd5640 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d7266115c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0974009930 tx=0x7f0974005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece46897db8fc911 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f096cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f096cff9640 1 -- 172.21.15.45:0/659216175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974035040 con 0x55d726613cb0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f096cff9640 1 -- 172.21.15.45:0/659216175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f097403d8a0 con 0x55d726613cb0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098001cb00 con 0x55d726613cb0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f096cff9640 1 -- 172.21.15.45:0/659216175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f097403daa0 con 0x55d726613cb0 2024-09-10T10:12:17.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f098e4d4740 10 client.15711 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.754+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d72634c620 con 0x55d726613cb0 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f096cff9640 1 -- 172.21.15.45:0/659216175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098001cb00 con 0x55d726613cb0 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f096cff9640 1 -- 172.21.15.45:0/659216175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f097403a040 con 0x55d726613cb0 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f098e4d4740 10 client.15711 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f098e4d4740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f098e4d4740 20 client.15711 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f098e4d4740 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d726623ac0 0x55d726625eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f098e4d4740 4 client.15711 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d72657aa20 con 0x55d726623ac0 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.755+0000 7f09894d6640 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d726623ac0 0x55d726625eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.756+0000 7f096cff9640 10 client.15711 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.756+0000 7f09894d6640 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d726623ac0 0x55d726625eb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15d4fc564ce973f server_cookie=78603b910a4e2ea2 in_seq=0 out_seq=0 2024-09-10T10:12:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.756+0000 7f096cff9640 1 -- 172.21.15.45:0/659216175 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d72657aa20 con 0x55d726623ac0 2024-09-10T10:12:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.756+0000 7f096cff9640 10 client.15711 handle_command_reply: tid=0 2024-09-10T10:12:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.756+0000 7f098e4d4740 1 client.15711 shutdown 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 msgr2=0x55d7266115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d7266115c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0974009930 tx=0x7f0974005b50 comp rx=0 tx=0).stop 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 shutdown_connections 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d726623ac0 0x55d726625eb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d7266115c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d726611080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f096cff9640 0 client.15711 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 --2- 172.21.15.45:0/659216175 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d726610b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 >> 172.21.15.45:0/659216175 conn(0x55d726605060 msgr2=0x55d72660adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 shutdown_connections 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 1 -- 172.21.15.45:0/659216175 wait complete. 2024-09-10T10:12:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.757+0000 7f098e4d4740 20 client.15711 trim_cache size 0 max 16384 2024-09-10T10:12:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 Processor -- start 2024-09-10T10:12:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 -- start start 2024-09-10T10:12:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d72660fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d7266113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d72662b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09841ab990 con 0x55d726613cb0 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f09841ae1f0 con 0x55d726609410 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f098e4d4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f09841abfe0 con 0x55d726608af0 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d72662b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d72662b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41904/0 (socket says 172.21.15.45:41904) 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 -- 172.21.15.45:0/2233890757 learned_addr learned my addr 172.21.15.45:0/2233890757 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d72660fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f09894d6640 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d7266113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 -- 172.21.15.45:0/2233890757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 msgr2=0x55d72660fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d72660fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 -- 172.21.15.45:0/2233890757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 msgr2=0x55d7266113f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d7266113f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 -- 172.21.15.45:0/2233890757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d726605cf0 con 0x55d726613cb0 2024-09-10T10:12:17.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f09894d6640 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d7266113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.772+0000 7f0988cd5640 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d72662b780 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f097403d430 tx=0x7f0974031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51780835f923e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f096effd640 1 -- 172.21.15.45:0/2233890757 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974035040 con 0x55d726613cb0 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f096effd640 1 -- 172.21.15.45:0/2233890757 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097403d8a0 con 0x55d726613cb0 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f096effd640 1 -- 172.21.15.45:0/2233890757 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974005360 con 0x55d726613cb0 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 -- 172.21.15.45:0/2233890757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 msgr2=0x55d72662b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d72662b780 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f097403d430 tx=0x7f0974031850 comp rx=0 tx=0).stop 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 -- 172.21.15.45:0/2233890757 shutdown_connections 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726613cb0 0x55d72662b780 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d7266113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 --2- 172.21.15.45:0/2233890757 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726608af0 0x55d72660fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 -- 172.21.15.45:0/2233890757 >> 172.21.15.45:0/2233890757 conn(0x55d726605060 msgr2=0x55d726612bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 -- 172.21.15.45:0/2233890757 shutdown_connections 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.773+0000 7f098e4d4740 1 -- 172.21.15.45:0/2233890757 wait complete. 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 -- start start 2024-09-10T10:12:17.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 Processor -- start 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726608af0 0x55d72661c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d72660d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d72660dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09841ac6b0 con 0x55d726608af0 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f09841adb20 con 0x55d726609410 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f09841acd80 con 0x55d726613cb0 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f098e4d4740 10 client.0 _ll_register_callbacks cb 0x7f0974009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726608af0 0x55d72661c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726608af0 0x55d72661c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41912/0 (socket says 172.21.15.45:41912) 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 -- 172.21.15.45:0/1364593706 learned_addr learned my addr 172.21.15.45:0/1364593706 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f09894d6640 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d72660d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 -- 172.21.15.45:0/1364593706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 msgr2=0x55d72660dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d72660dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 -- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 msgr2=0x55d72660d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d72660d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f0988cd5640 1 -- 172.21.15.45:0/1364593706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d72660b940 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.774+0000 7f09894d6640 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d72660d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f0988cd5640 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726608af0 0x55d72661c200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f09740059d0 tx=0x7f097402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b15929e7e4fb70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f096cff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f096cff9640 1 -- 172.21.15.45:0/1364593706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974035040 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f096cff9640 1 -- 172.21.15.45:0/1364593706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f09740052e0 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f097403d8a0 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f096cff9640 1 -- 172.21.15.45:0/1364593706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0974009520 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 10 client.15717 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d72668e330 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f096cff9640 1 -- 172.21.15.45:0/1364593706 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097403d8a0 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f096cff9640 1 -- 172.21.15.45:0/1364593706 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f097404a4b0 con 0x55d726608af0 2024-09-10T10:12:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 10 client.15717 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 20 client.15717 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d72668ea10 0x55d726690e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 4 client.15717 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.775+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d726574d70 con 0x55d72668ea10 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.776+0000 7f09894d6640 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d72668ea10 0x55d726690e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.776+0000 7f096cff9640 10 client.15717 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.776+0000 7f09894d6640 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d72668ea10 0x55d726690e30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dbf903a202e0094 server_cookie=d212783c2d012e in_seq=0 out_seq=0 2024-09-10T10:12:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.783+0000 7f096cff9640 1 -- 172.21.15.45:0/1364593706 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d726574d70 con 0x55d72668ea10 2024-09-10T10:12:17.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.783+0000 7f096cff9640 10 client.15717 handle_command_reply: tid=0 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.783+0000 7f098e4d4740 1 client.15717 shutdown 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726608af0 msgr2=0x55d72661c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726608af0 0x55d72661c200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f09740059d0 tx=0x7f097402a420 comp rx=0 tx=0).stop 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 shutdown_connections 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d72668ea10 0x55d726690e30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d726613cb0 0x55d72660dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f096cff9640 0 client.15717 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d726609410 0x55d72660d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 --2- 172.21.15.45:0/1364593706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d726608af0 0x55d72661c200 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 >> 172.21.15.45:0/1364593706 conn(0x55d726605060 msgr2=0x55d7266039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 shutdown_connections 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 1 -- 172.21.15.45:0/1364593706 wait complete. 2024-09-10T10:12:17.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098e4d4740 20 client.15717 trim_cache size 0 max 16384 2024-09-10T10:12:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:12:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:12:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:12:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:12:17.787 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f09700a2da0 msgr2=0x7f09700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f09700a2da0 0x7f09700a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f097c0ccea0 tx=0x7f097c06db80 comp rx=0 tx=0).stop 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984106a00 msgr2=0x7f098419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984106a00 0x7f098419cce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f097c00eaa0 tx=0x7f097c00ef60 comp rx=0 tx=0).stop 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 shutdown_connections 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0984106a00 0x7f098419cce0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f09700a2da0 0x7f09700a5260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0984101390 0x7f0984198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 --2- 172.21.15.45:0/706757081 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f09841009f0 0x7f0984198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.784+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 >> 172.21.15.45:0/706757081 conn(0x7f09840fc6b0 msgr2=0x7f098406a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.785+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 shutdown_connections 2024-09-10T10:12:17.788 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:17.785+0000 7f098b73a640 1 -- 172.21.15.45:0/706757081 wait complete. 2024-09-10T10:12:18.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:17 smithi162 ceph-mon[34963]: pgmap v281: 97 pgs: 97 active+clean; 41 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:12:18.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:17 smithi045 ceph-mon[31114]: pgmap v281: 97 pgs: 97 active+clean; 41 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:12:18.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:17 smithi047 ceph-mon[35244]: pgmap v281: 97 pgs: 97 active+clean; 41 GiB data, 136 GiB used, 937 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:12:20.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:19 smithi162 ceph-mon[34963]: pgmap v282: 97 pgs: 97 active+clean; 41 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 614 B/s wr, 2 op/s 2024-09-10T10:12:20.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:19 smithi045 ceph-mon[31114]: pgmap v282: 97 pgs: 97 active+clean; 41 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 614 B/s wr, 2 op/s 2024-09-10T10:12:20.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:19 smithi047 ceph-mon[35244]: pgmap v282: 97 pgs: 97 active+clean; 41 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 614 B/s wr, 2 op/s 2024-09-10T10:12:22.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:21 smithi162 ceph-mon[34963]: pgmap v283: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 499 B/s wr, 2 op/s 2024-09-10T10:12:22.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:21 smithi162 ceph-mon[34963]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:12:22.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:21 smithi045 ceph-mon[31114]: pgmap v283: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 499 B/s wr, 2 op/s 2024-09-10T10:12:22.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:21 smithi045 ceph-mon[31114]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:12:22.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:21 smithi047 ceph-mon[35244]: pgmap v283: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 499 B/s wr, 2 op/s 2024-09-10T10:12:22.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:21 smithi047 ceph-mon[35244]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T10:12:24.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:23 smithi162 ceph-mon[34963]: pgmap v285: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:24.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:23 smithi045 ceph-mon[31114]: pgmap v285: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:24.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:23 smithi047 ceph-mon[35244]: pgmap v285: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:26.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:25 smithi162 ceph-mon[34963]: pgmap v286: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:26.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:25 smithi045 ceph-mon[31114]: pgmap v286: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:26.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:25 smithi047 ceph-mon[35244]: pgmap v286: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:28.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:27 smithi162 ceph-mon[34963]: pgmap v287: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:28.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:27 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:12:28.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:27 smithi045 ceph-mon[31114]: pgmap v287: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:28.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:27 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:12:28.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:27 smithi047 ceph-mon[35244]: pgmap v287: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T10:12:28.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:27 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:12:29.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:28 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:12:29.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:28 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:12:29.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:28 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:12:29.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:28 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:12:29.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:28 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:12:29.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:28 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:12:29.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:28 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:12:29.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:28 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:12:29.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:28 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:12:30.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:29 smithi162 ceph-mon[34963]: pgmap v288: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:12:30.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:29 smithi045 ceph-mon[31114]: pgmap v288: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:12:30.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:29 smithi047 ceph-mon[35244]: pgmap v288: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 -- 172.21.15.45:0/3440808072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac103640 msgr2=0x7f78ac10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 --2- 172.21.15.45:0/3440808072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac103640 0x7f78ac10bb20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f789c009ca0 tx=0x7f789c02a950 comp rx=0 tx=0).stop 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 -- 172.21.15.45:0/3440808072 shutdown_connections 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 --2- 172.21.15.45:0/3440808072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac10c0f0 0x7f78ac10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 --2- 172.21.15.45:0/3440808072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac103640 0x7f78ac10bb20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 --2- 172.21.15.45:0/3440808072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78ac102d20 0x7f78ac103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 -- 172.21.15.45:0/3440808072 >> 172.21.15.45:0/3440808072 conn(0x7f78ac077990 msgr2=0x7f78ac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 -- 172.21.15.45:0/3440808072 shutdown_connections 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.886+0000 7f78b31cf640 1 -- 172.21.15.45:0/3440808072 wait complete. 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 Processor -- start 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 -- start start 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac102d20 0x7f78ac198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac103640 0x7f78ac198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78ac10c0f0 0x7f78ac19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac111ec0 con 0x7f78ac102d20 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac111d40 con 0x7f78ac103640 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b31cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac112040 con 0x7f78ac10c0f0 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b0f6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac102d20 0x7f78ac198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78abfff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac103640 0x7f78ac198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b0f6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac102d20 0x7f78ac198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50888/0 (socket says 172.21.15.45:50888) 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.887+0000 7f78b0f6b640 1 -- 172.21.15.45:0/1739437079 learned_addr learned my addr 172.21.15.45:0/1739437079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b176c640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78ac10c0f0 0x7f78ac19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b0f6b640 1 -- 172.21.15.45:0/1739437079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78ac10c0f0 msgr2=0x7f78ac19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78ac10c0f0 0x7f78ac19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b0f6b640 1 -- 172.21.15.45:0/1739437079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac103640 msgr2=0x7f78ac198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac103640 0x7f78ac198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b0f6b640 1 -- 172.21.15.45:0/1739437079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac19d340 con 0x7f78ac102d20 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78abfff640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac103640 0x7f78ac198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac102d20 0x7f78ac198350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f789400d770 tx=0x7f789400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:30.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78a9ffb640 1 -- 172.21.15.45:0/1739437079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894017070 con 0x7f78ac102d20 2024-09-10T10:12:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78a9ffb640 1 -- 172.21.15.45:0/1739437079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7894005500 con 0x7f78ac102d20 2024-09-10T10:12:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac19d630 con 0x7f78ac102d20 2024-09-10T10:12:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78a9ffb640 1 -- 172.21.15.45:0/1739437079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894004e00 con 0x7f78ac102d20 2024-09-10T10:12:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.888+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac1a4ec0 con 0x7f78ac102d20 2024-09-10T10:12:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.891+0000 7f78a9ffb640 1 -- 172.21.15.45:0/1739437079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7894004fa0 con 0x7f78ac102d20 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.891+0000 7f78b6043740 1 Processor -- start 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.891+0000 7f78b6043740 1 -- start start 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b6043740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6ed7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b6043740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b6043740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b6043740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fcd6e3c2f0 con 0x55fcd6ed7af0 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b6043740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fcd6de4270 con 0x55fcd6ee2cb0 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b6043740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fcd6e382b0 con 0x55fcd6ed8410 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78a9ffb640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f78980a2e20 0x7f78980a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6ed7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6ed7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50894/0 (socket says 172.21.15.45:50894) 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 -- 172.21.15.45:0/3724124158 learned_addr learned my addr 172.21.15.45:0/3724124158 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f78980a2e20 0x7f78980a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78abfff640 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78a9ffb640 1 -- 172.21.15.45:0/1739437079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f78940cee80 con 0x7f78ac102d20 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 -- 172.21.15.45:0/3724124158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 msgr2=0x55fcd6ee2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 -- 172.21.15.45:0/3724124158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 msgr2=0x55fcd6ee50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 -- 172.21.15.45:0/3724124158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd6de3340 con 0x55fcd6ed7af0 2024-09-10T10:12:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.892+0000 7f78b176c640 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6ed7ed0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f78a4014be0 tx=0x7f78a40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50498543547368f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b176c640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f78980a2e20 0x7f78980a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78a4011cb0 tx=0x7f78a4016000 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78a2ffd640 1 -- 172.21.15.45:0/3724124158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4022070 con 0x55fcd6ed7af0 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78a2ffd640 1 -- 172.21.15.45:0/3724124158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78a400c960 con 0x55fcd6ed7af0 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78a2ffd640 1 -- 172.21.15.45:0/3724124158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4004a00 con 0x55fcd6ed7af0 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b6043740 1 -- 172.21.15.45:0/3724124158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 msgr2=0x55fcd6ed7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b6043740 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6ed7ed0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f78a4014be0 tx=0x7f78a40117b0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b6043740 1 -- 172.21.15.45:0/3724124158 shutdown_connections 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b6043740 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b6043740 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b6043740 1 --2- 172.21.15.45:0/3724124158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6ed7ed0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.893+0000 7f78b6043740 1 -- 172.21.15.45:0/3724124158 >> 172.21.15.45:0/3724124158 conn(0x55fcd6ed4060 msgr2=0x55fcd6ed4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 -- 172.21.15.45:0/3724124158 shutdown_connections 2024-09-10T10:12:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 -- 172.21.15.45:0/3724124158 wait complete. 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 -- start start 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 Processor -- start 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fcd6ee5880 con 0x55fcd6ed8410 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55fcd6dc5290 con 0x55fcd6ed7af0 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55fcd6ee5a00 con 0x55fcd6ee2cb0 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.894+0000 7f78b6043740 10 client.0 _ll_register_callbacks cb 0x55fcd6df6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78b0f6b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78b0f6b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:51906/0 (socket says 172.21.15.45:51906) 2024-09-10T10:12:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50898/0 (socket says 172.21.15.45:50898) 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78b0f6b640 1 -- 172.21.15.45:0/1597391989 learned_addr learned my addr 172.21.15.45:0/1597391989 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78b176c640 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 -- 172.21.15.45:0/1597391989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 msgr2=0x55fcd6ee0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 -- 172.21.15.45:0/1597391989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 msgr2=0x55fcd6edfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 -- 172.21.15.45:0/1597391989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd6b4f210 con 0x55fcd6ed8410 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78b176c640 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78abfff640 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee0140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f789c02ae70 tx=0x7f789c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f59801bbc95aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f788bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f788bfff640 1 -- 172.21.15.45:0/1597391989 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c049060 con 0x55fcd6ed8410 2024-09-10T10:12:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f788bfff640 1 -- 172.21.15.45:0/1597391989 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f789c004040 con 0x55fcd6ed8410 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.895+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcd6e49a20 con 0x55fcd6ed8410 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f788bfff640 1 -- 172.21.15.45:0/1597391989 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f789c034040 con 0x55fcd6ed8410 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 10 client.15729 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fcd6c1b620 con 0x55fcd6ed8410 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f788bfff640 1 -- 172.21.15.45:0/1597391989 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fcd6e49a20 con 0x55fcd6ed8410 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f788bfff640 1 -- 172.21.15.45:0/1597391989 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f789c030900 con 0x55fcd6ed8410 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 10 client.15729 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 20 client.15729 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6ef2c70 0x55fcd6ef5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 4 client.15729 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.896+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcd6e43d70 con 0x55fcd6ef2c70 2024-09-10T10:12:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.897+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6ef2c70 0x55fcd6ef5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.897+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6ef2c70 0x55fcd6ef5060 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc7beff887d674f server_cookie=7b3cd96d42dba8cf in_seq=0 out_seq=0 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.897+0000 7f788bfff640 10 client.15729 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f788bfff640 1 -- 172.21.15.45:0/1597391989 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcd6e43d70 con 0x55fcd6ef2c70 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f788bfff640 10 client.15729 handle_command_reply: tid=0 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 client.15729 shutdown 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 msgr2=0x55fcd6ee0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee0140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f789c02ae70 tx=0x7f789c030000 comp rx=0 tx=0).stop 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 shutdown_connections 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6ef2c70 0x55fcd6ef5060 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6ee0140 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f788bfff640 0 client.15729 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 --2- 172.21.15.45:0/1597391989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 >> 172.21.15.45:0/1597391989 conn(0x55fcd6ed4060 msgr2=0x55fcd6ed9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 shutdown_connections 2024-09-10T10:12:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 1 -- 172.21.15.45:0/1597391989 wait complete. 2024-09-10T10:12:30.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.898+0000 7f78b6043740 20 client.15729 trim_cache size 0 max 16384 2024-09-10T10:12:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 Processor -- start 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 -- start start 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6eda560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1ab7a0 con 0x55fcd6ed7af0 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1ae000 con 0x55fcd6ee2cb0 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b6043740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1abdf0 con 0x55fcd6ed8410 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b0f6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b0f6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50906/0 (socket says 172.21.15.45:50906) 2024-09-10T10:12:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.913+0000 7f78b0f6b640 1 -- 172.21.15.45:0/675361163 learned_addr learned my addr 172.21.15.45:0/675361163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78abfff640 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6eda560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78b0f6b640 1 -- 172.21.15.45:0/675361163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 msgr2=0x55fcd6eda560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6eda560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78b176c640 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78b0f6b640 1 -- 172.21.15.45:0/675361163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 msgr2=0x55fcd6ee04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78b0f6b640 1 -- 172.21.15.45:0/675361163 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcd6ed5110 con 0x55fcd6ed7af0 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edeb20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f78940ccbb0 tx=0x7f789406b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd299501f547e7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78a27fc640 1 -- 172.21.15.45:0/675361163 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894020040 con 0x55fcd6ed7af0 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78a27fc640 1 -- 172.21.15.45:0/675361163 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f789409ba80 con 0x55fcd6ed7af0 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.914+0000 7f78a27fc640 1 -- 172.21.15.45:0/675361163 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894009cc0 con 0x55fcd6ed7af0 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 -- 172.21.15.45:0/675361163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 msgr2=0x55fcd6edeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edeb20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f78940ccbb0 tx=0x7f789406b1e0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 -- 172.21.15.45:0/675361163 shutdown_connections 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6ee04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ed8410 0x55fcd6eda560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 --2- 172.21.15.45:0/675361163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed7af0 0x55fcd6edeb20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 -- 172.21.15.45:0/675361163 >> 172.21.15.45:0/675361163 conn(0x55fcd6ed4060 msgr2=0x55fcd6ee1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 -- 172.21.15.45:0/675361163 shutdown_connections 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 -- 172.21.15.45:0/675361163 wait complete. 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 -- start start 2024-09-10T10:12:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.915+0000 7f78b6043740 1 Processor -- start 2024-09-10T10:12:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b6043740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6efa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b6043740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6efa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b6043740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6f58950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b6043740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1ac4c0 con 0x55fcd6ed8410 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b6043740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1ad930 con 0x55fcd6ed7af0 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b6043740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1acb90 con 0x55fcd6ee2cb0 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b6043740 10 client.0 _ll_register_callbacks cb 0x7f789c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6efa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6efa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50916/0 (socket says 172.21.15.45:50916) 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 -- 172.21.15.45:0/3410532265 learned_addr learned my addr 172.21.15.45:0/3410532265 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6efa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b176c640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6f58950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 -- 172.21.15.45:0/3410532265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 msgr2=0x55fcd6f58950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6f58950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 -- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 msgr2=0x55fcd6efa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6efa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 -- 172.21.15.45:0/3410532265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcd6f58ff0 con 0x55fcd6ed8410 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6efa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f78abfff640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6efa760 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f789c009bf0 tx=0x7f789c030ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1226a8db08fe88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f788bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.916+0000 7f788bfff640 1 -- 172.21.15.45:0/3410532265 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c049050 con 0x55fcd6ed8410 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f788bfff640 1 -- 172.21.15.45:0/3410532265 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f789c04d930 con 0x55fcd6ed8410 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f789409ba80 con 0x55fcd6ed8410 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f788bfff640 1 -- 172.21.15.45:0/3410532265 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f789c02f740 con 0x55fcd6ed8410 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 10 client.15735 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55fcd6f61590 con 0x55fcd6ed8410 2024-09-10T10:12:30.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f788bfff640 1 -- 172.21.15.45:0/3410532265 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f789409ba80 con 0x55fcd6ed8410 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f788bfff640 1 -- 172.21.15.45:0/3410532265 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f789c03a910 con 0x55fcd6ed8410 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 10 client.15735 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 20 client.15735 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6f61c80 0x55fcd6f640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 4 client.15735 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcd6dc0900 con 0x55fcd6f61c80 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.917+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6f61c80 0x55fcd6f640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.918+0000 7f788bfff640 10 client.15735 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.918+0000 7f78b0f6b640 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6f61c80 0x55fcd6f640a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c28e7fbb6b51 server_cookie=a4da0a693a5d73a3 in_seq=0 out_seq=0 2024-09-10T10:12:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.922+0000 7f788bfff640 1 -- 172.21.15.45:0/3410532265 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcd6dc0900 con 0x55fcd6f61c80 2024-09-10T10:12:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.922+0000 7f788bfff640 10 client.15735 handle_command_reply: tid=0 2024-09-10T10:12:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.922+0000 7f78b6043740 1 client.15735 shutdown 2024-09-10T10:12:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 msgr2=0x55fcd6efa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6efa760 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f789c009bf0 tx=0x7f789c030ac0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 shutdown_connections 2024-09-10T10:12:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55fcd6f61c80 0x55fcd6f640a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55fcd6ee2cb0 0x55fcd6f58950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fcd6ed8410 0x55fcd6efa760 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f788bfff640 0 client.15735 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 --2- 172.21.15.45:0/3410532265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55fcd6ed7af0 0x55fcd6efa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 >> 172.21.15.45:0/3410532265 conn(0x55fcd6ed4060 msgr2=0x55fcd6ee1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 shutdown_connections 2024-09-10T10:12:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 1 -- 172.21.15.45:0/3410532265 wait complete. 2024-09-10T10:12:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b6043740 20 client.15735 trim_cache size 0 max 16384 2024-09-10T10:12:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:12:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:12:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:12:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:12:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f78980a2e20 msgr2=0x7f78980a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b31cf640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f78980a2e20 0x7f78980a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78a4011cb0 tx=0x7f78a4016000 comp rx=0 tx=0).stop 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac102d20 msgr2=0x7f78ac198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b31cf640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac102d20 0x7f78ac198350 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f789400d770 tx=0x7f789400dc30 comp rx=0 tx=0).stop 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 shutdown_connections 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.923+0000 7f78b31cf640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f78ac10c0f0 0x7f78ac19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.924+0000 7f78b31cf640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f78980a2e20 0x7f78980a52e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.924+0000 7f78b31cf640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78ac103640 0x7f78ac198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.924+0000 7f78b31cf640 1 --2- 172.21.15.45:0/1739437079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78ac102d20 0x7f78ac198350 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.924+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 >> 172.21.15.45:0/1739437079 conn(0x7f78ac077990 msgr2=0x7f78ac0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.924+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 shutdown_connections 2024-09-10T10:12:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:30.924+0000 7f78b31cf640 1 -- 172.21.15.45:0/1739437079 wait complete. 2024-09-10T10:12:32.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:31 smithi162 ceph-mon[34963]: pgmap v289: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-10T10:12:32.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:31 smithi045 ceph-mon[31114]: pgmap v289: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-10T10:12:32.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:31 smithi047 ceph-mon[35244]: pgmap v289: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-10T10:12:34.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:33 smithi162 ceph-mon[34963]: pgmap v290: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-09-10T10:12:34.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:33 smithi045 ceph-mon[31114]: pgmap v290: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-09-10T10:12:34.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:33 smithi047 ceph-mon[35244]: pgmap v290: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-09-10T10:12:36.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:35 smithi162 ceph-mon[34963]: pgmap v291: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:36.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:35 smithi045 ceph-mon[31114]: pgmap v291: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:36.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:35 smithi047 ceph-mon[35244]: pgmap v291: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:38.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:37 smithi162 ceph-mon[34963]: pgmap v292: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:38.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:37 smithi045 ceph-mon[31114]: pgmap v292: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:38.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:37 smithi047 ceph-mon[35244]: pgmap v292: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:40.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:39 smithi162 ceph-mon[34963]: pgmap v293: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:40.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:39 smithi045 ceph-mon[31114]: pgmap v293: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:40.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:39 smithi047 ceph-mon[35244]: pgmap v293: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:42.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:41 smithi162 ceph-mon[34963]: pgmap v294: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:42.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:41 smithi045 ceph-mon[31114]: pgmap v294: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:42.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:41 smithi047 ceph-mon[35244]: pgmap v294: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 -- 172.21.15.45:0/962604573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc10c110 msgr2=0x7ff2fc10e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 --2- 172.21.15.45:0/962604573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc10c110 0x7ff2fc10e590 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009ca0 tx=0x7ff2f402a950 comp rx=0 tx=0).stop 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 -- 172.21.15.45:0/962604573 shutdown_connections 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 --2- 172.21.15.45:0/962604573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc10c110 0x7ff2fc10e590 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 --2- 172.21.15.45:0/962604573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc1035e0 0x7ff2fc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 --2- 172.21.15.45:0/962604573 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2fc102cc0 0x7ff2fc1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 -- 172.21.15.45:0/962604573 >> 172.21.15.45:0/962604573 conn(0x7ff2fc0fc6b0 msgr2=0x7ff2fc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 -- 172.21.15.45:0/962604573 shutdown_connections 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 -- 172.21.15.45:0/962604573 wait complete. 2024-09-10T10:12:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.024+0000 7ff3017b9640 1 Processor -- start 2024-09-10T10:12:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff3017b9640 1 -- start start 2024-09-10T10:12:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff3017b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc102cc0 0x7ff2fc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff3017b9640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc1035e0 0x7ff2fc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff3017b9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2fc10c110 0x7ff2fc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff3017b9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc111ef0 con 0x7ff2fc102cc0 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff3017b9640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc111d70 con 0x7ff2fc1035e0 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff3017b9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc112070 con 0x7ff2fc10c110 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc102cc0 0x7ff2fc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc102cc0 0x7ff2fc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53810/0 (socket says 172.21.15.45:53810) 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 -- 172.21.15.45:0/1514423075 learned_addr learned my addr 172.21.15.45:0/1514423075 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc1035e0 0x7ff2fc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2fc10c110 0x7ff2fc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 -- 172.21.15.45:0/1514423075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2fc10c110 msgr2=0x7ff2fc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2fc10c110 0x7ff2fc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 -- 172.21.15.45:0/1514423075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc1035e0 msgr2=0x7ff2fc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc1035e0 0x7ff2fc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2faffd640 1 -- 172.21.15.45:0/1514423075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc19f590 con 0x7ff2fc102cc0 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.025+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2fc10c110 0x7ff2fc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.026+0000 7ff2faffd640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc102cc0 0x7ff2fc19a5a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00e9f0 tx=0x7ff2ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.026+0000 7ff2e3fff640 1 -- 172.21.15.45:0/1514423075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00cd80 con 0x7ff2fc102cc0 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.026+0000 7ff2e3fff640 1 -- 172.21.15.45:0/1514423075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec005500 con 0x7ff2fc102cc0 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.026+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc19f880 con 0x7ff2fc102cc0 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.026+0000 7ff2e3fff640 1 -- 172.21.15.45:0/1514423075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec010690 con 0x7ff2fc102cc0 2024-09-10T10:12:44.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.026+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1a7110 con 0x7ff2fc102cc0 2024-09-10T10:12:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2e3fff640 1 -- 172.21.15.45:0/1514423075 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff2ec0046c0 con 0x7ff2fc102cc0 2024-09-10T10:12:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 Processor -- start 2024-09-10T10:12:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 -- start start 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f16f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f1720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56364f0c92f0 con 0x56364f165410 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56364f071270 con 0x56364f16fcb0 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff304626740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56364f0c52b0 con 0x56364f164af0 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f16f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f16f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53826/0 (socket says 172.21.15.45:53826) 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 -- 172.21.15.45:0/1339944376 learned_addr learned my addr 172.21.15.45:0/1339944376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f1720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 -- 172.21.15.45:0/1339944376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 msgr2=0x56364f164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f164ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 -- 172.21.15.45:0/1339944376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 msgr2=0x56364f1720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f1720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2faffd640 1 -- 172.21.15.45:0/1339944376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364f070340 con 0x56364f165410 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.029+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f1720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.030+0000 7ff2e3fff640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff2e40a2da0 0x7ff2e40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.030+0000 7ff2faffd640 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f16f770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0ccfe0 tx=0x7ff2ec0d5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01c178c282e23e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.030+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff2e40a2da0 0x7ff2e40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.030+0000 7ff2e3fff640 1 -- 172.21.15.45:0/1514423075 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff2ec0cf940 con 0x7ff2fc102cc0 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.030+0000 7ff2e17fa640 1 -- 172.21.15.45:0/1339944376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec09e030 con 0x56364f165410 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.030+0000 7ff2e17fa640 1 -- 172.21.15.45:0/1339944376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec09c540 con 0x56364f165410 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.030+0000 7ff2e17fa640 1 -- 172.21.15.45:0/1339944376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec09b980 con 0x56364f165410 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 -- 172.21.15.45:0/1339944376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 msgr2=0x56364f16f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff2e40a2da0 0x7ff2e40a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009430 tx=0x7ff2f4005d80 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f16f770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0ccfe0 tx=0x7ff2ec0d5ae0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 -- 172.21.15.45:0/1339944376 shutdown_connections 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f1720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f16f770 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 --2- 172.21.15.45:0/1339944376 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f164ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 -- 172.21.15.45:0/1339944376 >> 172.21.15.45:0/1339944376 conn(0x56364f161060 msgr2=0x56364f161440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 -- 172.21.15.45:0/1339944376 shutdown_connections 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 -- 172.21.15.45:0/1339944376 wait complete. 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 -- start start 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 Processor -- start 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f164af0 0x56364f16cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f165410 0x56364f16d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f16d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.031+0000 7ff304626740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56364f172880 con 0x56364f164af0 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff304626740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56364f052290 con 0x56364f16fcb0 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff304626740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x56364f172a00 con 0x56364f165410 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff304626740 10 client.0 _ll_register_callbacks cb 0x56364f083140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f164af0 0x56364f16cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f164af0 0x56364f16cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53828/0 (socket says 172.21.15.45:53828) 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 -- 172.21.15.45:0/944231603 learned_addr learned my addr 172.21.15.45:0/944231603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f165410 0x56364f16d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f16d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 -- 172.21.15.45:0/944231603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f165410 msgr2=0x56364f16d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f165410 0x56364f16d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 -- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 msgr2=0x56364f16d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f16d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 -- 172.21.15.45:0/944231603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364eddc210 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2faffd640 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f164af0 0x56364f16cca0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0cd670 tx=0x7ff2ec0d5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3934cf760f6e1816 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2deff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2deff5640 1 -- 172.21.15.45:0/944231603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec09e020 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2deff5640 1 -- 172.21.15.45:0/944231603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff2ec0984b0 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ec09c540 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff2deff5640 1 -- 172.21.15.45:0/944231603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec09b7a0 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff304626740 10 client.15747 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.032+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56364eea8620 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff2deff5640 1 -- 172.21.15.45:0/944231603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ec09c540 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff2deff5640 1 -- 172.21.15.45:0/944231603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff2ec09c0a0 con 0x56364f164af0 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff304626740 10 client.15747 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff304626740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff304626740 20 client.15747 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff304626740 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f17fba0 0x56364f181f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff304626740 4 client.15747 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56364f0d6a20 con 0x56364f17fba0 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.033+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f17fba0 0x56364f181f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f17fba0 0x56364f181f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d930a8aa2e63700f server_cookie=9ede13ae9046ac17 in_seq=0 out_seq=0 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff2deff5640 10 client.15747 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff2deff5640 1 -- 172.21.15.45:0/944231603 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56364f0d6a20 con 0x56364f17fba0 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff2deff5640 10 client.15747 handle_command_reply: tid=0 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff304626740 1 client.15747 shutdown 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f164af0 msgr2=0x56364f16cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff304626740 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f164af0 0x56364f16cca0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0cd670 tx=0x7ff2ec0d5730 comp rx=0 tx=0).stop 2024-09-10T10:12:44.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 shutdown_connections 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.034+0000 7ff304626740 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f17fba0 0x56364f181f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff304626740 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f16fcb0 0x56364f16d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff304626740 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f165410 0x56364f16d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff2deff5640 0 client.15747 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff304626740 1 --2- 172.21.15.45:0/944231603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f164af0 0x56364f16cca0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 >> 172.21.15.45:0/944231603 conn(0x56364f161060 msgr2=0x56364f166d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 shutdown_connections 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff304626740 1 -- 172.21.15.45:0/944231603 wait complete. 2024-09-10T10:12:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.035+0000 7ff304626740 20 client.15747 trim_cache size 0 max 16384 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.049+0000 7ff304626740 1 Processor -- start 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.049+0000 7ff304626740 1 -- start start 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff304626740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f16d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff304626740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f165410 0x56364f1875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff304626740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f16fcb0 0x56364f187b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff304626740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1adad0 con 0x56364f16fcb0 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff304626740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1b0330 con 0x56364f165410 2024-09-10T10:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff304626740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1ae120 con 0x56364f164af0 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f16fcb0 0x56364f187b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f16fcb0 0x56364f187b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53840/0 (socket says 172.21.15.45:53840) 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 -- 172.21.15.45:0/160646548 learned_addr learned my addr 172.21.15.45:0/160646548 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f165410 0x56364f1875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2faffd640 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f16d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 -- 172.21.15.45:0/160646548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 msgr2=0x56364f16d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f16d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 -- 172.21.15.45:0/160646548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f165410 msgr2=0x56364f1875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f165410 0x56364f1875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 -- 172.21.15.45:0/160646548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364f165240 con 0x56364f16fcb0 2024-09-10T10:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f165410 0x56364f1875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.050+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f16fcb0 0x56364f187b00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff2f402fc60 tx=0x7ff2f4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904640be27725074 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff2e0ff9640 1 -- 172.21.15.45:0/160646548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4034040 con 0x56364f16fcb0 2024-09-10T10:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff2e0ff9640 1 -- 172.21.15.45:0/160646548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4004040 con 0x56364f16fcb0 2024-09-10T10:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff2e0ff9640 1 -- 172.21.15.45:0/160646548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4030680 con 0x56364f16fcb0 2024-09-10T10:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 -- 172.21.15.45:0/160646548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f16fcb0 msgr2=0x56364f187b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f16fcb0 0x56364f187b00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff2f402fc60 tx=0x7ff2f4040000 comp rx=0 tx=0).stop 2024-09-10T10:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 -- 172.21.15.45:0/160646548 shutdown_connections 2024-09-10T10:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f16fcb0 0x56364f187b00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f165410 0x56364f1875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 --2- 172.21.15.45:0/160646548 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f164af0 0x56364f16d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 -- 172.21.15.45:0/160646548 >> 172.21.15.45:0/160646548 conn(0x56364f161060 msgr2=0x56364f16bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 -- 172.21.15.45:0/160646548 shutdown_connections 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 -- 172.21.15.45:0/160646548 wait complete. 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.051+0000 7ff304626740 1 -- start start 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 1 Processor -- start 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f164af0 0x56364f174070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f1745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f16fcb0 0x56364f1e9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1ae7f0 con 0x56364f165410 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1afc60 con 0x56364f164af0 2024-09-10T10:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1aeec0 con 0x56364f16fcb0 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff304626740 10 client.0 _ll_register_callbacks cb 0x7ff2ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f1745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f1745b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53848/0 (socket says 172.21.15.45:53848) 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 -- 172.21.15.45:0/321798777 learned_addr learned my addr 172.21.15.45:0/321798777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2faffd640 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f164af0 0x56364f174070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fb7fe640 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f16fcb0 0x56364f1e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 -- 172.21.15.45:0/321798777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f16fcb0 msgr2=0x56364f1e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f16fcb0 0x56364f1e9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 -- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f164af0 msgr2=0x56364f174070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f164af0 0x56364f174070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.052+0000 7ff2fa7fc640 1 -- 172.21.15.45:0/321798777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364f1e9ad0 con 0x56364f165410 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff2fa7fc640 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f1745b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0004700 tx=0x7ff2f000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f354827d9537bc82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff2deff5640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff2deff5640 1 -- 172.21.15.45:0/321798777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003d30 con 0x56364f165410 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff2deff5640 1 -- 172.21.15.45:0/321798777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7ff2f001d420 con 0x56364f165410 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f4004040 con 0x56364f165410 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff2deff5640 1 -- 172.21.15.45:0/321798777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff2f002d050 con 0x56364f165410 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff304626740 10 client.15753 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x56364f1eada0 con 0x56364f165410 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff2deff5640 1 -- 172.21.15.45:0/321798777 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4004040 con 0x56364f165410 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff2deff5640 1 -- 172.21.15.45:0/321798777 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7ff2f00118b0 con 0x56364f165410 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff304626740 10 client.15753 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff304626740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff304626740 20 client.15753 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.053+0000 7ff304626740 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f1eb520 0x56364f1ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.054+0000 7ff304626740 4 client.15753 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:12:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.054+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56364f0d0d70 con 0x56364f1eb520 2024-09-10T10:12:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.054+0000 7ff2faffd640 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f1eb520 0x56364f1ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.054+0000 7ff2deff5640 10 client.15753 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.054+0000 7ff2faffd640 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f1eb520 0x56364f1ed940 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33c2784d79fe49c server_cookie=5c1ea71a50604240 in_seq=0 out_seq=0 2024-09-10T10:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff2deff5640 1 -- 172.21.15.45:0/321798777 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56364f0d0d70 con 0x56364f1eb520 2024-09-10T10:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff2deff5640 10 client.15753 handle_command_reply: tid=0 2024-09-10T10:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 client.15753 shutdown 2024-09-10T10:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 msgr2=0x56364f1745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f1745b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0004700 tx=0x7ff2f000bd30 comp rx=0 tx=0).stop 2024-09-10T10:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 shutdown_connections 2024-09-10T10:12:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x56364f1eb520 0x56364f1ed940 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x56364f16fcb0 0x56364f1e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56364f165410 0x56364f1745b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 --2- 172.21.15.45:0/321798777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56364f164af0 0x56364f174070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 >> 172.21.15.45:0/321798777 conn(0x56364f161060 msgr2=0x56364f16b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.060+0000 7ff2deff5640 0 client.15753 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 shutdown_connections 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff304626740 1 -- 172.21.15.45:0/321798777 wait complete. 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff304626740 20 client.15753 trim_cache size 0 max 16384 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:12:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff2e40a2da0 msgr2=0x7ff2e40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff2e40a2da0 0x7ff2e40a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009430 tx=0x7ff2f4005d80 comp rx=0 tx=0).stop 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc102cc0 msgr2=0x7ff2fc19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc102cc0 0x7ff2fc19a5a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00e9f0 tx=0x7ff2ec00edf0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 shutdown_connections 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff2fc10c110 0x7ff2fc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7ff2e40a2da0 0x7ff2e40a5260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc1035e0 0x7ff2fc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 --2- 172.21.15.45:0/1514423075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2fc102cc0 0x7ff2fc19a5a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 >> 172.21.15.45:0/1514423075 conn(0x7ff2fc0fc6b0 msgr2=0x7ff2fc10d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 shutdown_connections 2024-09-10T10:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:44.061+0000 7ff3017b9640 1 -- 172.21.15.45:0/1514423075 wait complete. 2024-09-10T10:12:44.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:43 smithi162 ceph-mon[34963]: pgmap v295: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:44.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:43 smithi045 ceph-mon[31114]: pgmap v295: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:44.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:43 smithi047 ceph-mon[35244]: pgmap v295: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:46.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:45 smithi162 ceph-mon[34963]: pgmap v296: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:46.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:45 smithi045 ceph-mon[31114]: pgmap v296: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:46.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:45 smithi047 ceph-mon[35244]: pgmap v296: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:48.153 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:47 smithi162 ceph-mon[34963]: pgmap v297: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:12:48.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:47 smithi045 ceph-mon[31114]: pgmap v297: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:12:48.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:47 smithi047 ceph-mon[35244]: pgmap v297: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:12:50.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:49 smithi162 ceph-mon[34963]: pgmap v298: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:12:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:49 smithi045 ceph-mon[31114]: pgmap v298: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:12:50.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:49 smithi047 ceph-mon[35244]: pgmap v298: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-09-10T10:12:52.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:51 smithi162 ceph-mon[34963]: pgmap v299: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:51 smithi045 ceph-mon[31114]: pgmap v299: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:51 smithi047 ceph-mon[35244]: pgmap v299: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:54.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:53 smithi162 ceph-mon[34963]: pgmap v300: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:54.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:53 smithi045 ceph-mon[31114]: pgmap v300: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:54.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:53 smithi047 ceph-mon[35244]: pgmap v300: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:55 smithi162 ceph-mon[34963]: pgmap v301: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:56.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:55 smithi045 ceph-mon[31114]: pgmap v301: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:56.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:55 smithi047 ceph-mon[35244]: pgmap v301: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 -- 172.21.15.45:0/2665384265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c106a00 msgr2=0x7f7c4c106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 --2- 172.21.15.45:0/2665384265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c106a00 0x7f7c4c106de0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009930 tx=0x7f7c4802a420 comp rx=0 tx=0).stop 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 -- 172.21.15.45:0/2665384265 shutdown_connections 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 --2- 172.21.15.45:0/2665384265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c4c101390 0x7f7c4c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 --2- 172.21.15.45:0/2665384265 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c4c1009f0 0x7f7c4c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 --2- 172.21.15.45:0/2665384265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c106a00 0x7f7c4c106de0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 -- 172.21.15.45:0/2665384265 >> 172.21.15.45:0/2665384265 conn(0x7f7c4c0fc6b0 msgr2=0x7f7c4c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 -- 172.21.15.45:0/2665384265 shutdown_connections 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.161+0000 7f7c53111640 1 -- 172.21.15.45:0/2665384265 wait complete. 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 Processor -- start 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 -- start start 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c4c1009f0 0x7f7c4c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c101390 0x7f7c4c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c4c106a00 0x7f7c4c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c110e90 con 0x7f7c4c101390 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c110d10 con 0x7f7c4c1009f0 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c53111640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c111010 con 0x7f7c4c106a00 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.162+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c101390 0x7f7c4c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c101390 0x7f7c4c1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44892/0 (socket says 172.21.15.45:44892) 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 -- 172.21.15.45:0/664241903 learned_addr learned my addr 172.21.15.45:0/664241903 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c50ead640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c4c1009f0 0x7f7c4c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c516ae640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c4c106a00 0x7f7c4c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 -- 172.21.15.45:0/664241903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c4c106a00 msgr2=0x7f7c4c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c4c106a00 0x7f7c4c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 -- 172.21.15.45:0/664241903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c4c1009f0 msgr2=0x7f7c4c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c4c1009f0 0x7f7c4c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 -- 172.21.15.45:0/664241903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4c19c2a0 con 0x7f7c4c101390 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c43fff640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c101390 0x7f7c4c1977f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9f0 tx=0x7f7c3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c41ffb640 1 -- 172.21.15.45:0/664241903 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00cd80 con 0x7f7c4c101390 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c41ffb640 1 -- 172.21.15.45:0/664241903 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c005500 con 0x7f7c4c101390 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4c19c590 con 0x7f7c4c101390 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.163+0000 7f7c41ffb640 1 -- 172.21.15.45:0/664241903 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c010690 con 0x7f7c4c101390 2024-09-10T10:12:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.164+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4c1a3e20 con 0x7f7c4c101390 2024-09-10T10:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.166+0000 7f7c41ffb640 1 -- 172.21.15.45:0/664241903 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c3c0046c0 con 0x7f7c4c101390 2024-09-10T10:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.166+0000 7f7c55ea5740 1 Processor -- start 2024-09-10T10:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.166+0000 7f7c55ea5740 1 -- start start 2024-09-10T10:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c41ffb640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f7c340a2da0 0x7f7c340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368eb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b368432f0 con 0x558b368df2b0 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558b367eb270 con 0x558b368de990 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558b3683f2b0 con 0x558b368ebec0 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368eb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c516ae640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f7c340a2da0 0x7f7c340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c50ead640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368eb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44900/0 (socket says 172.21.15.45:44900) 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c50ead640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:36358/0 (socket says 172.21.15.45:36358) 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 -- 172.21.15.45:0/4146655094 learned_addr learned my addr 172.21.15.45:0/4146655094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c41ffb640 1 -- 172.21.15.45:0/664241903 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f7c3c0cfa10 con 0x7f7c4c101390 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c516ae640 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 -- 172.21.15.45:0/4146655094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 msgr2=0x558b368ee2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368ee2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 -- 172.21.15.45:0/4146655094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 msgr2=0x558b368ded70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368ded70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c43fff640 1 -- 172.21.15.45:0/4146655094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b367ea340 con 0x558b368df2b0 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c50ead640 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368ded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.167+0000 7f7c516ae640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f7c340a2da0 0x7f7c340a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005c40 tx=0x7f7c44012a90 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c43fff640 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368eb980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00c6a0 tx=0x7f7c3c098210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a738f4027b5688f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c32ffd640 1 -- 172.21.15.45:0/4146655094 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c09c5b0 con 0x558b368df2b0 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c32ffd640 1 -- 172.21.15.45:0/4146655094 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c09beb0 con 0x558b368df2b0 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c32ffd640 1 -- 172.21.15.45:0/4146655094 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c098520 con 0x558b368df2b0 2024-09-10T10:12:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c55ea5740 1 -- 172.21.15.45:0/4146655094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 msgr2=0x558b368eb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368eb980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00c6a0 tx=0x7f7c3c098210 comp rx=0 tx=0).stop 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c55ea5740 1 -- 172.21.15.45:0/4146655094 shutdown_connections 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368eb980 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/4146655094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368ded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.168+0000 7f7c55ea5740 1 -- 172.21.15.45:0/4146655094 >> 172.21.15.45:0/4146655094 conn(0x558b368db060 msgr2=0x558b368db440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 -- 172.21.15.45:0/4146655094 shutdown_connections 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 -- 172.21.15.45:0/4146655094 wait complete. 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 -- start start 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 Processor -- start 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368df2b0 0x558b368ef420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558b368eea90 con 0x558b368de990 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558b367cc290 con 0x558b368df2b0 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.169+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x558b368eec10 con 0x558b368ebec0 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c55ea5740 10 client.0 _ll_register_callbacks cb 0x558b367fd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44904/0 (socket says 172.21.15.45:44904) 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 -- 172.21.15.45:0/2290784864 learned_addr learned my addr 172.21.15.45:0/2290784864 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c516ae640 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c43fff640 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368df2b0 0x558b368ef420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 -- 172.21.15.45:0/2290784864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 msgr2=0x558b368f9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 -- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368df2b0 msgr2=0x558b368ef420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368df2b0 0x558b368ef420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 -- 172.21.15.45:0/2290784864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b36556210 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c50ead640 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368eeee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7c4802aa10 tx=0x7f7c48005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be56c0c5e9bbdbaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c2bfff640 1 -- 172.21.15.45:0/2290784864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48035040 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c3c09beb0 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.170+0000 7f7c2bfff640 1 -- 172.21.15.45:0/2290784864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f7c480048c0 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c2bfff640 1 -- 172.21.15.45:0/2290784864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7c48005d50 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 10 client.15765 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558b36622620 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c2bfff640 1 -- 172.21.15.45:0/2290784864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c3c09beb0 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c2bfff640 1 -- 172.21.15.45:0/2290784864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7c48003c60 con 0x558b368de990 2024-09-10T10:12:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 10 client.15765 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 20 client.15765 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b368fe0e0 0x558b369004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 4 client.15765 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b36850a20 con 0x558b368fe0e0 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.171+0000 7f7c43fff640 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b368fe0e0 0x558b369004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.172+0000 7f7c2bfff640 10 client.15765 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.172+0000 7f7c43fff640 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b368fe0e0 0x558b369004d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623b63c24938c915 server_cookie=d93d7aafc190f633 in_seq=0 out_seq=0 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.172+0000 7f7c2bfff640 1 -- 172.21.15.45:0/2290784864 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b36850a20 con 0x558b368fe0e0 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.172+0000 7f7c2bfff640 10 client.15765 handle_command_reply: tid=0 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.172+0000 7f7c55ea5740 1 client.15765 shutdown 2024-09-10T10:12:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 msgr2=0x558b368eeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368eeee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7c4802aa10 tx=0x7f7c48005270 comp rx=0 tx=0).stop 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 shutdown_connections 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b368fe0e0 0x558b369004d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368df2b0 0x558b368ef420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/2290784864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368eeee0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c2bfff640 0 client.15765 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 >> 172.21.15.45:0/2290784864 conn(0x558b368db060 msgr2=0x558b368da1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 shutdown_connections 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 1 -- 172.21.15.45:0/2290784864 wait complete. 2024-09-10T10:12:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.173+0000 7f7c55ea5740 20 client.15765 trim_cache size 0 max 16384 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.187+0000 7f7c55ea5740 1 Processor -- start 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.187+0000 7f7c55ea5740 1 -- start start 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368df2b0 0x558b368e9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368ebec0 0x558b36905d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1a96a0 con 0x558b368de990 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1abf00 con 0x558b368ebec0 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1a9cf0 con 0x558b368df2b0 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368e99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44918/0 (socket says 172.21.15.45:44918) 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 -- 172.21.15.45:0/3669176474 learned_addr learned my addr 172.21.15.45:0/3669176474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c516ae640 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368ebec0 0x558b36905d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c43fff640 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368df2b0 0x558b368e9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 -- 172.21.15.45:0/3669176474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368df2b0 msgr2=0x558b368e9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368df2b0 0x558b368e9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 -- 172.21.15.45:0/3669176474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368ebec0 msgr2=0x558b36905d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368ebec0 0x558b36905d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 -- 172.21.15.45:0/3669176474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b368df0e0 con 0x558b368de990 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c43fff640 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368df2b0 0x558b368e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c516ae640 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368ebec0 0x558b36905d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c50ead640 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368e99a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7c48006010 tx=0x7f7c4802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46bb185c2fba8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.188+0000 7f7c327fc640 1 -- 172.21.15.45:0/3669176474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48052070 con 0x558b368de990 2024-09-10T10:12:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c327fc640 1 -- 172.21.15.45:0/3669176474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48040070 con 0x558b368de990 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c327fc640 1 -- 172.21.15.45:0/3669176474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48035040 con 0x558b368de990 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 -- 172.21.15.45:0/3669176474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 msgr2=0x558b368e99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368e99a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7c48006010 tx=0x7f7c4802a420 comp rx=0 tx=0).stop 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 -- 172.21.15.45:0/3669176474 shutdown_connections 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368ebec0 0x558b36905d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368df2b0 0x558b368e9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/3669176474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368de990 0x558b368e99a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 -- 172.21.15.45:0/3669176474 >> 172.21.15.45:0/3669176474 conn(0x558b368db060 msgr2=0x558b368eee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 -- 172.21.15.45:0/3669176474 shutdown_connections 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 -- 172.21.15.45:0/3669176474 wait complete. 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.189+0000 7f7c55ea5740 1 -- start start 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 1 Processor -- start 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368f1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1aa3c0 con 0x558b368df2b0 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1ab830 con 0x558b368de990 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f7c4c1aaa90 con 0x558b368ebec0 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c55ea5740 10 client.0 _ll_register_callbacks cb 0x7f7c3c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368f1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368f1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44934/0 (socket says 172.21.15.45:44934) 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 -- 172.21.15.45:0/1203428355 learned_addr learned my addr 172.21.15.45:0/1203428355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c516ae640 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c50ead640 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 -- 172.21.15.45:0/1203428355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 msgr2=0x558b368f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 -- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 msgr2=0x558b368f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.190+0000 7f7c43fff640 1 -- 172.21.15.45:0/1203428355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b36967de0 con 0x558b368df2b0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c43fff640 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368f1240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c098310 tx=0x7f7c3c09bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b788702ddef56c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c2bfff640 1 -- 172.21.15.45:0/1203428355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c093020 con 0x558b368df2b0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c2bfff640 1 -- 172.21.15.45:0/1203428355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f7c3c098520 con 0x558b368df2b0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c48040070 con 0x558b368df2b0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c2bfff640 1 -- 172.21.15.45:0/1203428355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f7c3c09beb0 con 0x558b368df2b0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 10 client.15771 fetch_fsmap learned FSMap version 21 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x558b369690d0 con 0x558b368df2b0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c2bfff640 1 -- 172.21.15.45:0/1203428355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c48040070 con 0x558b368df2b0 2024-09-10T10:12:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c2bfff640 1 -- 172.21.15.45:0/1203428355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f7c3c096020 con 0x558b368df2b0 2024-09-10T10:12:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 10 client.15771 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:12:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:12:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 20 client.15771 populate_metadata read hostname 'smithi045' 2024-09-10T10:12:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b36969430 0x558b3696b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:12:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 4 client.15771 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:12:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.191+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b3684ad70 con 0x558b36969430 2024-09-10T10:12:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.192+0000 7f7c50ead640 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b36969430 0x558b3696b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.192+0000 7f7c50ead640 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b36969430 0x558b3696b850 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920f700f9ed0b458 server_cookie=ef1776af1e87a2f8 in_seq=0 out_seq=0 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.192+0000 7f7c2bfff640 10 client.15771 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c2bfff640 1 -- 172.21.15.45:0/1203428355 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b3684ad70 con 0x558b36969430 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c2bfff640 10 client.15771 handle_command_reply: tid=0 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 client.15771 shutdown 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 msgr2=0x558b368f1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368f1240 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c098310 tx=0x7f7c3c09bd40 comp rx=0 tx=0).stop 2024-09-10T10:12:57.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 shutdown_connections 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x558b36969430 0x558b3696b850 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x558b368ebec0 0x558b368f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558b368df2b0 0x558b368f1240 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 --2- 172.21.15.45:0/1203428355 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558b368de990 0x558b368f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c2bfff640 0 client.15771 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 >> 172.21.15.45:0/1203428355 conn(0x558b368db060 msgr2=0x558b368e9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 shutdown_connections 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 1 -- 172.21.15.45:0/1203428355 wait complete. 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.197+0000 7f7c55ea5740 20 client.15771 trim_cache size 0 max 16384 2024-09-10T10:12:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:12:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:12:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:12:57.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f7c340a2da0 msgr2=0x7f7c340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f7c340a2da0 0x7f7c340a5260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005c40 tx=0x7f7c44012a90 comp rx=0 tx=0).stop 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c101390 msgr2=0x7f7c4c1977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c101390 0x7f7c4c1977f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9f0 tx=0x7f7c3c00edf0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 shutdown_connections 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7c4c106a00 0x7f7c4c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7c4c101390 0x7f7c4c1977f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f7c340a2da0 0x7f7c340a5260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 --2- 172.21.15.45:0/664241903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c4c1009f0 0x7f7c4c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 >> 172.21.15.45:0/664241903 conn(0x7f7c4c0fc6b0 msgr2=0x7f7c4c0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:12:57.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 shutdown_connections 2024-09-10T10:12:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:12:57.198+0000 7f7c53111640 1 -- 172.21.15.45:0/664241903 wait complete. 2024-09-10T10:12:58.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:57 smithi162 ceph-mon[34963]: pgmap v302: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:58.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:57 smithi045 ceph-mon[31114]: pgmap v302: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:12:58.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:57 smithi047 ceph-mon[35244]: pgmap v302: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:00.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:12:59 smithi162 ceph-mon[34963]: pgmap v303: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-10T10:13:00.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:12:59 smithi045 ceph-mon[31114]: pgmap v303: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-10T10:13:00.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:12:59 smithi047 ceph-mon[35244]: pgmap v303: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-09-10T10:13:02.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:01 smithi162 ceph-mon[34963]: pgmap v304: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-09-10T10:13:02.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:01 smithi045 ceph-mon[31114]: pgmap v304: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-09-10T10:13:02.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:01 smithi047 ceph-mon[35244]: pgmap v304: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-09-10T10:13:04.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:03 smithi162 ceph-mon[34963]: pgmap v305: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:04.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:03 smithi045 ceph-mon[31114]: pgmap v305: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:04.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:03 smithi047 ceph-mon[35244]: pgmap v305: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:06.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:05 smithi162 ceph-mon[34963]: pgmap v306: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:06.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:05 smithi045 ceph-mon[31114]: pgmap v306: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:06.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:05 smithi047 ceph-mon[35244]: pgmap v306: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:08.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:07 smithi162 ceph-mon[34963]: pgmap v307: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:08.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:07 smithi045 ceph-mon[31114]: pgmap v307: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:08.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:07 smithi047 ceph-mon[35244]: pgmap v307: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:10.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:09 smithi047 ceph-mon[35244]: pgmap v308: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.338+0000 7fb834523640 1 -- 172.21.15.45:0/127495879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1009d0 msgr2=0x7fb82c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.338+0000 7fb834523640 1 --2- 172.21.15.45:0/127495879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1009d0 0x7fb82c100e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009860 tx=0x7fb81c02a340 comp rx=0 tx=0).stop 2024-09-10T10:13:10.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.340+0000 7fb834523640 1 -- 172.21.15.45:0/127495879 shutdown_connections 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.340+0000 7fb834523640 1 --2- 172.21.15.45:0/127495879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb82c101370 0x7fb82c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.340+0000 7fb834523640 1 --2- 172.21.15.45:0/127495879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1009d0 0x7fb82c100e30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.340+0000 7fb834523640 1 --2- 172.21.15.45:0/127495879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb82c1069e0 0x7fb82c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.340+0000 7fb834523640 1 -- 172.21.15.45:0/127495879 >> 172.21.15.45:0/127495879 conn(0x7fb82c0fc6b0 msgr2=0x7fb82c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.340+0000 7fb834523640 1 -- 172.21.15.45:0/127495879 shutdown_connections 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.340+0000 7fb834523640 1 -- 172.21.15.45:0/127495879 wait complete. 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.341+0000 7fb834523640 1 Processor -- start 2024-09-10T10:13:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.341+0000 7fb834523640 1 -- start start 2024-09-10T10:13:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.342+0000 7fb834523640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb82c1009d0 0x7fb82c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.342+0000 7fb834523640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb82c101370 0x7fb82c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.342+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb82c1009d0 0x7fb82c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.343+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb82c1009d0 0x7fb82c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:41784/0 (socket says 172.21.15.45:41784) 2024-09-10T10:13:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.343+0000 7fb8322bf640 1 -- 172.21.15.45:0/1951190390 learned_addr learned my addr 172.21.15.45:0/1951190390 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.343+0000 7fb834523640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1069e0 0x7fb82c19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.343+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c110e30 con 0x7fb82c101370 2024-09-10T10:13:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.343+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c110cb0 con 0x7fb82c1069e0 2024-09-10T10:13:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.343+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c110fb0 con 0x7fb82c1009d0 2024-09-10T10:13:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.343+0000 7fb832ac0640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1069e0 0x7fb82c19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.344+0000 7fb832ac0640 1 -- 172.21.15.45:0/1951190390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb82c1009d0 msgr2=0x7fb82c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.344+0000 7fb832ac0640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb82c1009d0 0x7fb82c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.344+0000 7fb832ac0640 1 -- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb82c101370 msgr2=0x7fb82c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.344+0000 7fb832ac0640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb82c101370 0x7fb82c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.344+0000 7fb832ac0640 1 -- 172.21.15.45:0/1951190390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c19c2d0 con 0x7fb82c1069e0 2024-09-10T10:13:10.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.348+0000 7fb832ac0640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1069e0 0x7fb82c19bba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb82400ef00 tx=0x7fb82400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.351+0000 7fb8237fe640 1 -- 172.21.15.45:0/1951190390 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824004b20 con 0x7fb82c1069e0 2024-09-10T10:13:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.351+0000 7fb8237fe640 1 -- 172.21.15.45:0/1951190390 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb824003e50 con 0x7fb82c1069e0 2024-09-10T10:13:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.352+0000 7fb8237fe640 1 -- 172.21.15.45:0/1951190390 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82400b570 con 0x7fb82c1069e0 2024-09-10T10:13:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.352+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c19c5c0 con 0x7fb82c1069e0 2024-09-10T10:13:10.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.352+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c1a3e50 con 0x7fb82c1069e0 2024-09-10T10:13:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.356+0000 7fb8372bb740 1 Processor -- start 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.357+0000 7fb8237fe640 1 -- 172.21.15.45:0/1951190390 <== mon.2 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb82401d020 con 0x7fb82c1069e0 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.358+0000 7fb8372bb740 1 -- start start 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.358+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 0x560fd628c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.358+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd627f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.358+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6293af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.358+0000 7fb8372bb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560fd61e22f0 con 0x560fd628bc20 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.358+0000 7fb8372bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560fd618a270 con 0x560fd627f580 2024-09-10T10:13:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.358+0000 7fb8372bb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x560fd61de2b0 con 0x560fd627ebe0 2024-09-10T10:13:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.359+0000 7fb8237fe640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8180a2f50 0x7fb8180a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.359+0000 7fb8237fe640 1 -- 172.21.15.45:0/1951190390 <== mon.2 v2:172.21.15.47:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8600+0+0 (secure 0 0 0) 0x7fb8240cf3c0 con 0x7fb82c1069e0 2024-09-10T10:13:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.360+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd627f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.360+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd627f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:41786/0 (socket says 172.21.15.45:41786) 2024-09-10T10:13:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.360+0000 7fb8322bf640 1 -- 172.21.15.45:0/3604688829 learned_addr learned my addr 172.21.15.45:0/3604688829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb831abe640 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6293af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb8322bf640 1 -- 172.21.15.45:0/3604688829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 msgr2=0x560fd6293af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb8322bf640 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6293af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb8322bf640 1 -- 172.21.15.45:0/3604688829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 msgr2=0x560fd628c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb8322bf640 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 0x560fd628c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb8322bf640 1 -- 172.21.15.45:0/3604688829 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd6189340 con 0x560fd627ebe0 2024-09-10T10:13:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb8322bf640 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd627f040 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb82800bd10 tx=0x7fb82800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b62c2bbd434d8dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.361+0000 7fb832ac0640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8180a2f50 0x7fb8180a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.362+0000 7fb820bfe640 1 -- 172.21.15.45:0/3604688829 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828015070 con 0x560fd627ebe0 2024-09-10T10:13:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.362+0000 7fb820bfe640 1 -- 172.21.15.45:0/3604688829 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8280035f0 con 0x560fd627ebe0 2024-09-10T10:13:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.363+0000 7fb8372bb740 1 -- 172.21.15.45:0/3604688829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 msgr2=0x560fd627f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.363+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd627f040 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb82800bd10 tx=0x7fb82800f730 comp rx=0 tx=0).stop 2024-09-10T10:13:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.363+0000 7fb8372bb740 1 -- 172.21.15.45:0/3604688829 shutdown_connections 2024-09-10T10:13:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.363+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6293af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.363+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd627f040 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.363+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3604688829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 0x560fd628c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.363+0000 7fb8372bb740 1 -- 172.21.15.45:0/3604688829 >> 172.21.15.45:0/3604688829 conn(0x560fd627b720 msgr2=0x560fd6279aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:10.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.364+0000 7fb8372bb740 1 -- 172.21.15.45:0/3604688829 shutdown_connections 2024-09-10T10:13:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.364+0000 7fb8372bb740 1 -- 172.21.15.45:0/3604688829 wait complete. 2024-09-10T10:13:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 -- start start 2024-09-10T10:13:10.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 Processor -- start 2024-09-10T10:13:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627ebe0 0x560fd6290f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627f580 0x560fd62914c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 0x560fd6291a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560fd62a51b0 con 0x560fd627f580 2024-09-10T10:13:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560fd616b290 con 0x560fd627ebe0 2024-09-10T10:13:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x560fd62a5330 con 0x560fd628bc20 2024-09-10T10:13:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.365+0000 7fb8372bb740 10 client.0 _ll_register_callbacks cb 0x560fd619c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627f580 0x560fd62914c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627f580 0x560fd62914c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48608/0 (socket says 172.21.15.45:48608) 2024-09-10T10:13:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 -- 172.21.15.45:0/3158749862 learned_addr learned my addr 172.21.15.45:0/3158749862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb831abe640 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627ebe0 0x560fd6290f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb831abe640 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 0x560fd6291a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb832ac0640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8180a2f50 0x7fb8180a5410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8240ccb70 tx=0x7fb82400cdf0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 -- 172.21.15.45:0/3158749862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 msgr2=0x560fd6291a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 0x560fd6291a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 -- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627ebe0 msgr2=0x560fd6290f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627ebe0 0x560fd6290f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.369+0000 7fb8322bf640 1 -- 172.21.15.45:0/3158749862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd5ef5210 con 0x560fd627f580 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.370+0000 7fb8322bf640 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627f580 0x560fd62914c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb82800b6e0 tx=0x7fb82800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60014a12694849ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.370+0000 7fb815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:13:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.370+0000 7fb815ffb640 1 -- 172.21.15.45:0/3158749862 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828015070 con 0x560fd627f580 2024-09-10T10:13:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.371+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8280035f0 con 0x560fd627f580 2024-09-10T10:13:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.371+0000 7fb815ffb640 1 -- 172.21.15.45:0/3158749862 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8600+0+0 (secure 0 0 0) 0x7fb8280039b0 con 0x560fd627f580 2024-09-10T10:13:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.371+0000 7fb815ffb640 1 -- 172.21.15.45:0/3158749862 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb828003fe0 con 0x560fd627f580 2024-09-10T10:13:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.371+0000 7fb8372bb740 10 client.15777 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.371+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560fd5fc1620 con 0x560fd627f580 2024-09-10T10:13:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.372+0000 7fb815ffb640 1 -- 172.21.15.45:0/3158749862 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8280035f0 con 0x560fd627f580 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.373+0000 7fb815ffb640 1 -- 172.21.15.45:0/3158749862 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb828011900 con 0x560fd627f580 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.374+0000 7fb8372bb740 10 client.15777 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.374+0000 7fb8372bb740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.374+0000 7fb8372bb740 20 client.15777 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.374+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd62b2810 0x560fd62b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.374+0000 7fb8372bb740 4 client.15777 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.374+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fd61efa20 con 0x560fd62b2810 2024-09-10T10:13:10.376 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.374+0000 7fb831abe640 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd62b2810 0x560fd62b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.375+0000 7fb831abe640 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd62b2810 0x560fd62b4cb0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77e52ce0ee6128b server_cookie=2e5514706e0e2cd8 in_seq=0 out_seq=0 2024-09-10T10:13:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.375+0000 7fb815ffb640 10 client.15777 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.376+0000 7fb815ffb640 1 -- 172.21.15.45:0/3158749862 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fd61efa20 con 0x560fd62b2810 2024-09-10T10:13:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.376+0000 7fb815ffb640 10 client.15777 handle_command_reply: tid=0 2024-09-10T10:13:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.376+0000 7fb8372bb740 1 client.15777 shutdown 2024-09-10T10:13:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.376+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627f580 msgr2=0x560fd62914c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.376+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627f580 0x560fd62914c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb82800b6e0 tx=0x7fb82800be00 comp rx=0 tx=0).stop 2024-09-10T10:13:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.380+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 shutdown_connections 2024-09-10T10:13:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.380+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd62b2810 0x560fd62b4cb0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.380+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 0x560fd6291a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.380+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627f580 0x560fd62914c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.380+0000 7fb8372bb740 1 --2- 172.21.15.45:0/3158749862 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627ebe0 0x560fd6290f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.380+0000 7fb815ffb640 0 client.15777 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.380+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 >> 172.21.15.45:0/3158749862 conn(0x560fd627b720 msgr2=0x560fd628d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.381+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 shutdown_connections 2024-09-10T10:13:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.381+0000 7fb8372bb740 1 -- 172.21.15.45:0/3158749862 wait complete. 2024-09-10T10:13:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.383+0000 7fb8372bb740 20 client.15777 trim_cache size 0 max 16384 2024-09-10T10:13:10.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:09 smithi162 ceph-mon[34963]: pgmap v308: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:10.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:09 smithi045 ceph-mon[31114]: pgmap v308: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-10T10:13:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.421+0000 7fb8372bb740 1 Processor -- start 2024-09-10T10:13:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.422+0000 7fb8372bb740 1 -- start start 2024-09-10T10:13:10.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.422+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627ebe0 0x560fd627cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.422+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6291830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.422+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 0x560fd627d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.422+0000 7fb8372bb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c0712c0 con 0x560fd627ebe0 2024-09-10T10:13:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.422+0000 7fb8372bb740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c073a90 con 0x560fd627f580 2024-09-10T10:13:10.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.422+0000 7fb8372bb740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c071880 con 0x560fd628bc20 2024-09-10T10:13:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6291830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6291830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:35566/0 (socket says 172.21.15.45:35566) 2024-09-10T10:13:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 -- 172.21.15.45:0/1929231020 learned_addr learned my addr 172.21.15.45:0/1929231020 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 -- 172.21.15.45:0/1929231020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 msgr2=0x560fd627d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 --2- 172.21.15.45:0/1929231020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 0x560fd627d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 -- 172.21.15.45:0/1929231020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627ebe0 msgr2=0x560fd627cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 --2- 172.21.15.45:0/1929231020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627ebe0 0x560fd627cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 -- 172.21.15.45:0/1929231020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fd628c370 con 0x560fd627f580 2024-09-10T10:13:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.426+0000 7fb8322bf640 1 --2- 172.21.15.45:0/1929231020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6291830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8280058c0 tx=0x7fb82800b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5656062cfa45378 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.427+0000 7fb817fff640 1 -- 172.21.15.45:0/1929231020 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828015070 con 0x560fd627f580 2024-09-10T10:13:10.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.427+0000 7fb817fff640 1 -- 172.21.15.45:0/1929231020 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb828003dc0 con 0x560fd627f580 2024-09-10T10:13:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.427+0000 7fb817fff640 1 -- 172.21.15.45:0/1929231020 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82801d040 con 0x560fd627f580 2024-09-10T10:13:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.428+0000 7fb8372bb740 1 -- 172.21.15.45:0/1929231020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 msgr2=0x560fd6291830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.428+0000 7fb8372bb740 1 --2- 172.21.15.45:0/1929231020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6291830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8280058c0 tx=0x7fb82800b820 comp rx=0 tx=0).stop 2024-09-10T10:13:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.429+0000 7fb8372bb740 1 -- 172.21.15.45:0/1929231020 shutdown_connections 2024-09-10T10:13:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.429+0000 7fb8372bb740 1 --2- 172.21.15.45:0/1929231020 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd628bc20 0x560fd627d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.429+0000 7fb8372bb740 1 --2- 172.21.15.45:0/1929231020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd6291830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.429+0000 7fb8372bb740 1 --2- 172.21.15.45:0/1929231020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd627ebe0 0x560fd627cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.429+0000 7fb8372bb740 1 -- 172.21.15.45:0/1929231020 >> 172.21.15.45:0/1929231020 conn(0x560fd627b720 msgr2=0x560fd628a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.431+0000 7fb8372bb740 1 -- 172.21.15.45:0/1929231020 shutdown_connections 2024-09-10T10:13:10.433 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.431+0000 7fb8372bb740 1 -- 172.21.15.45:0/1929231020 wait complete. 2024-09-10T10:13:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.432+0000 7fb8372bb740 1 -- start start 2024-09-10T10:13:10.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.432+0000 7fb8372bb740 1 Processor -- start 2024-09-10T10:13:10.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd62aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb8372bb740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd62aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb831abe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd62aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb831abe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd62aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:41812/0 (socket says 172.21.15.45:41812) 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb831abe640 1 -- 172.21.15.45:0/405738319 learned_addr learned my addr 172.21.15.45:0/405738319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb8372bb740 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 0x560fd62b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb8322bf640 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd62aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c071f50 con 0x560fd628bc20 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c0733c0 con 0x560fd627f580 2024-09-10T10:13:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.433+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c072620 con 0x560fd627ebe0 2024-09-10T10:13:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.434+0000 7fb8372bb740 10 client.0 _ll_register_callbacks cb 0x7fb82800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.435+0000 7fb8322bf640 1 -- 172.21.15.45:0/405738319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 msgr2=0x560fd62aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.435+0000 7fb8322bf640 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd62aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.435+0000 7fb8322bf640 1 -- 172.21.15.45:0/405738319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 msgr2=0x560fd62b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.435+0000 7fb8322bf640 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 0x560fd62b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.437 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.435+0000 7fb8322bf640 1 -- 172.21.15.45:0/405738319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fd62b9d20 con 0x560fd627f580 2024-09-10T10:13:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb8322bf640 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd62aab10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x560fd624f7e0 tx=0x7fb828003be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0b0b768e12537d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb815ffb640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-09-10T10:13:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb815ffb640 1 -- 172.21.15.45:0/405738319 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828015040 con 0x560fd627f580 2024-09-10T10:13:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb815ffb640 1 -- 172.21.15.45:0/405738319 <== mon.2 v2:172.21.15.47:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8600+0+0 (secure 0 0 0) 0x7fb828024630 con 0x560fd627f580 2024-09-10T10:13:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb828003dc0 con 0x560fd627f580 2024-09-10T10:13:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb815ffb640 1 -- 172.21.15.45:0/405738319 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fb8280035f0 con 0x560fd627f580 2024-09-10T10:13:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb8372bb740 10 client.25394 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.436+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560fd631ce40 con 0x560fd627f580 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb815ffb640 1 -- 172.21.15.45:0/405738319 <== mon.2 v2:172.21.15.47:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb828003dc0 con 0x560fd627f580 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb815ffb640 1 -- 172.21.15.45:0/405738319 <== mon.2 v2:172.21.15.47:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fb82803c580 con 0x560fd627f580 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb8372bb740 10 client.25394 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb8372bb740 10 client.25394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb8372bb740 20 client.25394 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb8372bb740 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd631d4b0 0x560fd631f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb8372bb740 4 client.25394 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:13:10.440 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.437+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fd61e9d70 con 0x560fd631d4b0 2024-09-10T10:13:10.443 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.441+0000 7fb831abe640 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd631d4b0 0x560fd631f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.444+0000 7fb831abe640 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd631d4b0 0x560fd631f970 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a360419ffb6a8d06 server_cookie=bfff61b92ed6fe41 in_seq=0 out_seq=0 2024-09-10T10:13:10.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.444+0000 7fb815ffb640 10 client.25394 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.475+0000 7fb815ffb640 1 -- 172.21.15.45:0/405738319 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fd61e9d70 con 0x560fd631d4b0 2024-09-10T10:13:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.475+0000 7fb815ffb640 10 client.25394 handle_command_reply: tid=0 2024-09-10T10:13:10.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.476+0000 7fb8372bb740 1 client.25394 shutdown 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 msgr2=0x560fd62aab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd62aab10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x560fd624f7e0 tx=0x7fb828003be0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 shutdown_connections 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x560fd631d4b0 0x560fd631f970 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560fd628bc20 0x560fd62b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb815ffb640 0 client.25394 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560fd627f580 0x560fd62aab10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 --2- 172.21.15.45:0/405738319 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x560fd627ebe0 0x560fd62aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.477+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 >> 172.21.15.45:0/405738319 conn(0x560fd627b720 msgr2=0x560fd627d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.478+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 shutdown_connections 2024-09-10T10:13:10.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.478+0000 7fb8372bb740 1 -- 172.21.15.45:0/405738319 wait complete. 2024-09-10T10:13:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.480+0000 7fb8372bb740 20 client.25394 trim_cache size 0 max 16384 2024-09-10T10:13:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:13:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:13:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:13:10.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:13:10.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8180a2f50 msgr2=0x7fb8180a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8180a2f50 0x7fb8180a5410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8240ccb70 tx=0x7fb82400cdf0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1069e0 msgr2=0x7fb82c19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1069e0 0x7fb82c19bba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb82400ef00 tx=0x7fb82400c740 comp rx=0 tx=0).stop 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 shutdown_connections 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb8180a2f50 0x7fb8180a5410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb82c1069e0 0x7fb82c19bba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb82c101370 0x7fb82c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 --2- 172.21.15.45:0/1951190390 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb82c1009d0 0x7fb82c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 >> 172.21.15.45:0/1951190390 conn(0x7fb82c0fc6b0 msgr2=0x7fb82c0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 shutdown_connections 2024-09-10T10:13:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:10.482+0000 7fb834523640 1 -- 172.21.15.45:0/1951190390 wait complete. 2024-09-10T10:13:11.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:10 smithi162 ceph-mon[34963]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:13:11.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:10 smithi045 ceph-mon[31114]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:13:11.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:10 smithi047 ceph-mon[35244]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T10:13:12.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:11 smithi162 ceph-mon[34963]: pgmap v310: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-09-10T10:13:12.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:11 smithi045 ceph-mon[31114]: pgmap v310: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-09-10T10:13:12.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:11 smithi047 ceph-mon[35244]: pgmap v310: 97 pgs: 97 active+clean; 41 GiB data, 127 GiB used, 945 GiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-09-10T10:13:14.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:13 smithi162 ceph-mon[34963]: pgmap v311: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 40 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-09-10T10:13:14.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:13 smithi045 ceph-mon[31114]: pgmap v311: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 40 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-09-10T10:13:14.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:13 smithi047 ceph-mon[35244]: pgmap v311: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 40 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-09-10T10:13:16.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:15 smithi162 ceph-mon[34963]: pgmap v312: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 32 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:16.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:15 smithi045 ceph-mon[31114]: pgmap v312: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 32 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:16.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:15 smithi047 ceph-mon[35244]: pgmap v312: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 32 GiB data, 112 GiB used, 961 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:18.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:17 smithi045 ceph-mon[31114]: pgmap v313: 97 pgs: 97 active+clean; 31 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:18.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:17 smithi162 ceph-mon[34963]: pgmap v313: 97 pgs: 97 active+clean; 31 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:18.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:17 smithi047 ceph-mon[35244]: pgmap v313: 97 pgs: 97 active+clean; 31 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:20.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:19 smithi162 ceph-mon[34963]: pgmap v314: 97 pgs: 97 active+clean; 31 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:20.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:19 smithi045 ceph-mon[31114]: pgmap v314: 97 pgs: 97 active+clean; 31 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:20.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:19 smithi047 ceph-mon[35244]: pgmap v314: 97 pgs: 97 active+clean; 31 GiB data, 106 GiB used, 967 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T10:13:22.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:21 smithi162 ceph-mon[34963]: pgmap v315: 97 pgs: 97 active+clean; 31 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 907 B/s wr, 2 op/s 2024-09-10T10:13:22.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:21 smithi162 ceph-mon[34963]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:13:22.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:21 smithi045 ceph-mon[31114]: pgmap v315: 97 pgs: 97 active+clean; 31 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 907 B/s wr, 2 op/s 2024-09-10T10:13:22.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:21 smithi045 ceph-mon[31114]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:13:22.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:21 smithi047 ceph-mon[35244]: pgmap v315: 97 pgs: 97 active+clean; 31 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 907 B/s wr, 2 op/s 2024-09-10T10:13:22.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:21 smithi047 ceph-mon[35244]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T10:13:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/3260564687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f0106cb0 msgr2=0x7fa4f010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/3260564687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f0106cb0 0x7fa4f010e830 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009900 tx=0x7fa4e802a410 comp rx=0 tx=0).stop 2024-09-10T10:13:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/3260564687 shutdown_connections 2024-09-10T10:13:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/3260564687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f0106cb0 0x7fa4f010e830 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/3260564687 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f0100a50 0x7fa4f0106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/3260564687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4f01000a0 0x7fa4f0100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/3260564687 >> 172.21.15.45:0/3260564687 conn(0x7fa4f0076ff0 msgr2=0x7fa4f00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/3260564687 shutdown_connections 2024-09-10T10:13:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.590+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/3260564687 wait complete. 2024-09-10T10:13:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.591+0000 7fa4f5b9f640 1 Processor -- start 2024-09-10T10:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.591+0000 7fa4f5b9f640 1 -- start start 2024-09-10T10:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f5b9f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f01000a0 0x7fa4f01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f5b9f640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4f0106cb0 0x7fa4f0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f5b9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f019caa0 0x7fa4f019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f5b9f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0111d10 con 0x7fa4f019caa0 2024-09-10T10:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f5b9f640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0111b90 con 0x7fa4f0106cb0 2024-09-10T10:13:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f5b9f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0111e90 con 0x7fa4f01000a0 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f019caa0 0x7fa4f019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f019caa0 0x7fa4f019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39078/0 (socket says 172.21.15.45:39078) 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 -- 172.21.15.45:0/745546255 learned_addr learned my addr 172.21.15.45:0/745546255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f01000a0 0x7fa4f01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4effff640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4f0106cb0 0x7fa4f0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 -- 172.21.15.45:0/745546255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f01000a0 msgr2=0x7fa4f01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f01000a0 0x7fa4f01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 -- 172.21.15.45:0/745546255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4f0106cb0 msgr2=0x7fa4f0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4f0106cb0 0x7fa4f0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 -- 172.21.15.45:0/745546255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f019d590 con 0x7fa4f019caa0 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f539e640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f019caa0 0x7fa4f019cf50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8005ab0 tx=0x7fa4e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4edffb640 1 -- 172.21.15.45:0/745546255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e803f030 con 0x7fa4f019caa0 2024-09-10T10:13:23.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4edffb640 1 -- 172.21.15.45:0/745546255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e8003ce0 con 0x7fa4f019caa0 2024-09-10T10:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.592+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f019d820 con 0x7fa4f019caa0 2024-09-10T10:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.593+0000 7fa4edffb640 1 -- 172.21.15.45:0/745546255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8035070 con 0x7fa4f019caa0 2024-09-10T10:13:23.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.593+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f01a5040 con 0x7fa4f019caa0 2024-09-10T10:13:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.595+0000 7fa4edffb640 1 -- 172.21.15.45:0/745546255 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa4e8042030 con 0x7fa4f019caa0 2024-09-10T10:13:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.595+0000 7fa4f9c73740 1 Processor -- start 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.595+0000 7fa4f9c73740 1 -- start start 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce12910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce129b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce129be80 0x555ce129e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555ce11f52f0 con 0x555ce1290cc0 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555ce119d270 con 0x555ce12915e0 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x555ce11f12b0 con 0x555ce129be80 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4edffb640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa4dc0a2d70 0x7fa4dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce12910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce12910a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39092/0 (socket says 172.21.15.45:39092) 2024-09-10T10:13:23.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 -- 172.21.15.45:0/3906370384 learned_addr learned my addr 172.21.15.45:0/3906370384 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f539e640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa4dc0a2d70 0x7fa4dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce129b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce129be80 0x555ce129e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4edffb640 1 -- 172.21.15.45:0/745546255 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa4e80efbb0 con 0x7fa4f019caa0 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 -- 172.21.15.45:0/3906370384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce129be80 msgr2=0x555ce129e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce129be80 0x555ce129e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 -- 172.21.15.45:0/3906370384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 msgr2=0x555ce129b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce129b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.596+0000 7fa4effff640 1 -- 172.21.15.45:0/3906370384 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce119c340 con 0x555ce1290cc0 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.597+0000 7fa4effff640 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce12910a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8005f40 tx=0x7fa4d8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb4f85ffb6c60ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.597+0000 7fa4f539e640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa4dc0a2d70 0x7fa4dc0a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80bbe00 tx=0x7fa4e808e0c0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:23.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.597+0000 7fa4e6ffd640 1 -- 172.21.15.45:0/3906370384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d801a020 con 0x555ce1290cc0 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.597+0000 7fa4e6ffd640 1 -- 172.21.15.45:0/3906370384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d8012df0 con 0x555ce1290cc0 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.597+0000 7fa4e6ffd640 1 -- 172.21.15.45:0/3906370384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8010770 con 0x555ce1290cc0 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.597+0000 7fa4f9c73740 1 -- 172.21.15.45:0/3906370384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 msgr2=0x555ce12910a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.597+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce12910a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8005f40 tx=0x7fa4d8009b30 comp rx=0 tx=0).stop 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 -- 172.21.15.45:0/3906370384 shutdown_connections 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce129be80 0x555ce129e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce129b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/3906370384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce12910a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 -- 172.21.15.45:0/3906370384 >> 172.21.15.45:0/3906370384 conn(0x555ce128d230 msgr2=0x555ce128d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 -- 172.21.15.45:0/3906370384 shutdown_connections 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 -- 172.21.15.45:0/3906370384 wait complete. 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 -- start start 2024-09-10T10:13:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.598+0000 7fa4f9c73740 1 Processor -- start 2024-09-10T10:13:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce1298d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce1299280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce129be80 0x555ce12997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555ce129ea50 con 0x555ce1290cc0 2024-09-10T10:13:23.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555ce117e290 con 0x555ce129be80 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x555ce129ebd0 con 0x555ce12915e0 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f9c73740 10 client.0 _ll_register_callbacks cb 0x555ce11af140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce1298d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce1298d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39096/0 (socket says 172.21.15.45:39096) 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 -- 172.21.15.45:0/652399371 learned_addr learned my addr 172.21.15.45:0/652399371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce1299280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce129be80 0x555ce12997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 -- 172.21.15.45:0/652399371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 msgr2=0x555ce1299280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce1299280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 -- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce129be80 msgr2=0x555ce12997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce129be80 0x555ce12997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 -- 172.21.15.45:0/652399371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce0f08210 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4effff640 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce1298d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8012fe0 tx=0x7fa4d80098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd3193f70f21f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4cffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4cffff640 1 -- 172.21.15.45:0/652399371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8029020 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.599+0000 7fa4cffff640 1 -- 172.21.15.45:0/652399371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa4d8004a40 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d8012df0 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4cffff640 1 -- 172.21.15.45:0/652399371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa4d801a070 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 10 client.15789 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555ce0fd4620 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4cffff640 1 -- 172.21.15.45:0/652399371 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d8012df0 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4cffff640 1 -- 172.21.15.45:0/652399371 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa4d800f040 con 0x555ce1290cc0 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 10 client.15789 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:23.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 20 client.15789 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce12abd40 0x555ce12ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 4 client.15789 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ce1202a20 con 0x555ce12abd40 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.600+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce12abd40 0x555ce12ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.601+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce12abd40 0x555ce12ae130 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec2e7a060e1ed5a server_cookie=14aeb75262508c37 in_seq=0 out_seq=0 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.601+0000 7fa4cffff640 10 client.15789 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.601+0000 7fa4cffff640 1 -- 172.21.15.45:0/652399371 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ce1202a20 con 0x555ce12abd40 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.601+0000 7fa4cffff640 10 client.15789 handle_command_reply: tid=0 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.601+0000 7fa4f9c73740 1 client.15789 shutdown 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 msgr2=0x555ce1298d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce1298d40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8012fe0 tx=0x7fa4d80098a0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 shutdown_connections 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce12abd40 0x555ce12ae130 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce129be80 0x555ce12997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4cffff640 0 client.15789 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce1299280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/652399371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce1290cc0 0x555ce1298d40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 >> 172.21.15.45:0/652399371 conn(0x555ce128d230 msgr2=0x555ce1292f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 shutdown_connections 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 1 -- 172.21.15.45:0/652399371 wait complete. 2024-09-10T10:13:23.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.602+0000 7fa4f9c73740 20 client.15789 trim_cache size 0 max 16384 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.616+0000 7fa4f9c73740 1 Processor -- start 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.616+0000 7fa4f9c73740 1 -- start start 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce1290cc0 0x555ce12995f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce12b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01ab6a0 con 0x555ce129be80 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01adf00 con 0x555ce1290cc0 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01abcf0 con 0x555ce12915e0 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce12b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce12b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.45:58632/0 (socket says 172.21.15.45:58632) 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 -- 172.21.15.45:0/2683140149 learned_addr learned my addr 172.21.15.45:0/2683140149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39100/0 (socket says 172.21.15.45:39100) 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4effff640 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce1290cc0 0x555ce12995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 -- 172.21.15.45:0/2683140149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce1290cc0 msgr2=0x555ce12995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce1290cc0 0x555ce12995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 -- 172.21.15.45:0/2683140149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 msgr2=0x555ce12b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 -- 172.21.15.45:0/2683140149 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ce129b890 con 0x555ce12915e0 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.617+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce12b3760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000b890 tx=0x7fa4e000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf349cead18501f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4e67fc640 1 -- 172.21.15.45:0/2683140149 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0003d00 con 0x555ce12915e0 2024-09-10T10:13:23.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4e67fc640 1 -- 172.21.15.45:0/2683140149 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0027950 con 0x555ce12915e0 2024-09-10T10:13:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4e67fc640 1 -- 172.21.15.45:0/2683140149 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0011910 con 0x555ce12915e0 2024-09-10T10:13:23.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 -- 172.21.15.45:0/2683140149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 msgr2=0x555ce12b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce12b3760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000b890 tx=0x7fa4e000b390 comp rx=0 tx=0).stop 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 -- 172.21.15.45:0/2683140149 shutdown_connections 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce12915e0 0x555ce12b3760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/2683140149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce1290cc0 0x555ce12995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 -- 172.21.15.45:0/2683140149 >> 172.21.15.45:0/2683140149 conn(0x555ce128d230 msgr2=0x555ce129adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 -- 172.21.15.45:0/2683140149 shutdown_connections 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.618+0000 7fa4f9c73740 1 -- 172.21.15.45:0/2683140149 wait complete. 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 -- start start 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 Processor -- start 2024-09-10T10:13:23.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce1290cc0 0x555ce1295c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce12961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01ac3c0 con 0x555ce129be80 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01ad830 con 0x555ce12915e0 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01aca90 con 0x555ce1290cc0 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f9c73740 10 client.0 _ll_register_callbacks cb 0x7fa4e000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4effff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39108/0 (socket says 172.21.15.45:39108) 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4effff640 1 -- 172.21.15.45:0/1052991139 learned_addr learned my addr 172.21.15.45:0/1052991139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce12961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.619+0000 7fa4effff640 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce1290cc0 0x555ce1295c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4effff640 1 -- 172.21.15.45:0/1052991139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce1290cc0 msgr2=0x555ce1295c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4effff640 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce1290cc0 0x555ce1295c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4effff640 1 -- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 msgr2=0x555ce12961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4effff640 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce12961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4effff640 1 -- 172.21.15.45:0/1052991139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ce12a22f0 con 0x555ce129be80 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4effff640 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12966f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80096c0 tx=0x7fa4d8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6569990e5a113ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4cffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4cffff640 1 -- 172.21.15.45:0/1052991139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8029040 con 0x555ce129be80 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4cffff640 1 -- 172.21.15.45:0/1052991139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa4d8003b70 con 0x555ce129be80 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e0027950 con 0x555ce129be80 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4cffff640 1 -- 172.21.15.45:0/1052991139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa4d8003d70 con 0x555ce129be80 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4f9c73740 10 client.15795 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x555ce13169c0 con 0x555ce129be80 2024-09-10T10:13:23.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.620+0000 7fa4cffff640 1 -- 172.21.15.45:0/1052991139 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0027950 con 0x555ce129be80 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4cffff640 1 -- 172.21.15.45:0/1052991139 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa4d8022af0 con 0x555ce129be80 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f9c73740 10 client.15795 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f9c73740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f9c73740 20 client.15795 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce1317010 0x555ce1319400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f9c73740 4 client.15795 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ce11fcd70 con 0x555ce1317010 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce1317010 0x555ce1319400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4cffff640 10 client.15795 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:23.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.621+0000 7fa4f4b9d640 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce1317010 0x555ce1319400 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49bd967354cba87a server_cookie=273ae08ff7ef452a in_seq=0 out_seq=0 2024-09-10T10:13:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4cffff640 1 -- 172.21.15.45:0/1052991139 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ce11fcd70 con 0x555ce1317010 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4cffff640 10 client.15795 handle_command_reply: tid=0 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 client.15795 shutdown 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 msgr2=0x555ce12966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12966f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80096c0 tx=0x7fa4d8005f90 comp rx=0 tx=0).stop 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 shutdown_connections 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x555ce1317010 0x555ce1319400 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555ce129be80 0x555ce12966f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555ce12915e0 0x555ce12961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 --2- 172.21.15.45:0/1052991139 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x555ce1290cc0 0x555ce1295c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4cffff640 0 client.15795 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.626+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 >> 172.21.15.45:0/1052991139 conn(0x555ce128d230 msgr2=0x555ce129a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:23.629 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 shutdown_connections 2024-09-10T10:13:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f9c73740 1 -- 172.21.15.45:0/1052991139 wait complete. 2024-09-10T10:13:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f9c73740 20 client.15795 trim_cache size 0 max 16384 2024-09-10T10:13:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:13:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:13:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:13:23.630 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa4dc0a2d70 msgr2=0x7fa4dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa4dc0a2d70 0x7fa4dc0a5230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa4e80bbe00 tx=0x7fa4e808e0c0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f019caa0 msgr2=0x7fa4f019cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f019caa0 0x7fa4f019cf50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8005ab0 tx=0x7fa4e80042f0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 shutdown_connections 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4f019caa0 0x7fa4f019cf50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa4dc0a2d70 0x7fa4dc0a5230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4f0106cb0 0x7fa4f0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.631 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 --2- 172.21.15.45:0/745546255 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa4f01000a0 0x7fa4f01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 >> 172.21.15.45:0/745546255 conn(0x7fa4f0076ff0 msgr2=0x7fa4f0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 shutdown_connections 2024-09-10T10:13:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:23.627+0000 7fa4f5b9f640 1 -- 172.21.15.45:0/745546255 wait complete. 2024-09-10T10:13:24.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:23 smithi162 ceph-mon[34963]: pgmap v317: 97 pgs: 97 active+clean; 31 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 921 B/s wr, 2 op/s 2024-09-10T10:13:24.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:23 smithi045 ceph-mon[31114]: pgmap v317: 97 pgs: 97 active+clean; 31 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 921 B/s wr, 2 op/s 2024-09-10T10:13:24.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:23 smithi047 ceph-mon[35244]: pgmap v317: 97 pgs: 97 active+clean; 31 GiB data, 99 GiB used, 973 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 921 B/s wr, 2 op/s 2024-09-10T10:13:26.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:25 smithi162 ceph-mon[34963]: pgmap v318: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-09-10T10:13:26.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:25 smithi045 ceph-mon[31114]: pgmap v318: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-09-10T10:13:26.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:25 smithi047 ceph-mon[35244]: pgmap v318: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-09-10T10:13:28.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:27 smithi162 ceph-mon[34963]: pgmap v319: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-09-10T10:13:28.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:27 smithi045 ceph-mon[31114]: pgmap v319: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-09-10T10:13:28.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:27 smithi047 ceph-mon[35244]: pgmap v319: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 102 B/s wr, 2 op/s 2024-09-10T10:13:29.243 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:28 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:13:29.249 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:28 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:13:29.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:28 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:13:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:29 smithi162 ceph-mon[34963]: pgmap v320: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:13:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:29 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:29 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:29 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:29 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:29 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:29 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:29 smithi045 ceph-mon[31114]: pgmap v320: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:13:30.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:29 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:29 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:29 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:29 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:29 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:29 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:29 smithi047 ceph-mon[35244]: pgmap v320: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:13:30.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:29 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:29 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:29 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:29 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:29 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:30.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:29 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:31.301 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.302 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.303 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.303 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.303 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:13:31.303 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:13:31.303 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:30 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:31.311 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.311 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.311 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.311 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:13:31.312 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:30 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:13:31.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:13:31.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:13:31.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:30 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:13:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:31 smithi162 ceph-mon[34963]: pgmap v321: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:13:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:31 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-10T10:13:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:31 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi162 to 176.0M 2024-09-10T10:13:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:31 smithi162 ceph-mon[34963]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-10T10:13:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:31 smithi162 ceph-mon[34963]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-10T10:13:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:31 smithi162 ceph-mon[34963]: Unable to set osd_memory_target on smithi162 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T10:13:32.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:31 smithi162 ceph-mon[34963]: Unable to set osd_memory_target on smithi045 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-10T10:13:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:31 smithi045 ceph-mon[31114]: pgmap v321: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:13:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:31 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-10T10:13:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:31 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi162 to 176.0M 2024-09-10T10:13:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:31 smithi045 ceph-mon[31114]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-10T10:13:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:31 smithi045 ceph-mon[31114]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-10T10:13:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:31 smithi045 ceph-mon[31114]: Unable to set osd_memory_target on smithi162 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T10:13:32.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:31 smithi045 ceph-mon[31114]: Unable to set osd_memory_target on smithi045 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-10T10:13:32.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:31 smithi047 ceph-mon[35244]: pgmap v321: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-10T10:13:32.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:31 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi047 to 177.0M 2024-09-10T10:13:32.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:31 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi162 to 176.0M 2024-09-10T10:13:32.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:31 smithi047 ceph-mon[35244]: Adjusting osd_memory_target on smithi045 to 177.0M 2024-09-10T10:13:32.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:31 smithi047 ceph-mon[35244]: Unable to set osd_memory_target on smithi047 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-10T10:13:32.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:31 smithi047 ceph-mon[35244]: Unable to set osd_memory_target on smithi162 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-09-10T10:13:32.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:31 smithi047 ceph-mon[35244]: Unable to set osd_memory_target on smithi045 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-10T10:13:34.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:33 smithi162 ceph-mon[34963]: pgmap v322: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 284 B/s wr, 2 op/s 2024-09-10T10:13:34.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:33 smithi045 ceph-mon[31114]: pgmap v322: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 284 B/s wr, 2 op/s 2024-09-10T10:13:34.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:33 smithi047 ceph-mon[35244]: pgmap v322: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.0 KiB/s rd, 284 B/s wr, 2 op/s 2024-09-10T10:13:36.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:35 smithi162 ceph-mon[34963]: pgmap v323: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:36.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:35 smithi045 ceph-mon[31114]: pgmap v323: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:36.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:35 smithi047 ceph-mon[35244]: pgmap v323: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 -- 172.21.15.45:0/362315686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c107b70 msgr2=0x7fd55c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 --2- 172.21.15.45:0/362315686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c107b70 0x7fd55c107f50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009930 tx=0x7fd54c02a420 comp rx=0 tx=0).stop 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 -- 172.21.15.45:0/362315686 shutdown_connections 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 --2- 172.21.15.45:0/362315686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd55c102470 0x7fd55c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 --2- 172.21.15.45:0/362315686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c101ad0 0x7fd55c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 --2- 172.21.15.45:0/362315686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c107b70 0x7fd55c107f50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 -- 172.21.15.45:0/362315686 >> 172.21.15.45:0/362315686 conn(0x7fd55c0fd7d0 msgr2=0x7fd55c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 -- 172.21.15.45:0/362315686 shutdown_connections 2024-09-10T10:13:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.728+0000 7fd5608a4640 1 -- 172.21.15.45:0/362315686 wait complete. 2024-09-10T10:13:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 Processor -- start 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 -- start start 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c101ad0 0x7fd55c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c102470 0x7fd55c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd55c107b70 0x7fd55c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c111f20 con 0x7fd55c102470 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c111da0 con 0x7fd55c101ad0 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.729+0000 7fd5608a4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1120a0 con 0x7fd55c107b70 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c102470 0x7fd55c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c102470 0x7fd55c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50500/0 (socket says 172.21.15.45:50500) 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 -- 172.21.15.45:0/939478503 learned_addr learned my addr 172.21.15.45:0/939478503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd55c107b70 0x7fd55c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd55a59c640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c101ad0 0x7fd55c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 -- 172.21.15.45:0/939478503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd55c107b70 msgr2=0x7fd55c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd55c107b70 0x7fd55c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 -- 172.21.15.45:0/939478503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c101ad0 msgr2=0x7fd55c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c101ad0 0x7fd55c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 -- 172.21.15.45:0/939478503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55c19d3a0 con 0x7fd55c102470 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd55a59c640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c101ad0 0x7fd55c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd559d9b640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c102470 0x7fd55c1988f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9f0 tx=0x7fd55000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd54b7fe640 1 -- 172.21.15.45:0/939478503 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55000cd80 con 0x7fd55c102470 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd54b7fe640 1 -- 172.21.15.45:0/939478503 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550010910 con 0x7fd55c102470 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd54b7fe640 1 -- 172.21.15.45:0/939478503 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550018990 con 0x7fd55c102470 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55c19d690 con 0x7fd55c102470 2024-09-10T10:13:36.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.730+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd55c1a4f20 con 0x7fd55c102470 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd54b7fe640 1 -- 172.21.15.45:0/939478503 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd550010ac0 con 0x7fd55c102470 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 Processor -- start 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 -- start start 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca161b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1620c0 0x55d1ca1644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d1ca0b72f0 con 0x55d1ca1620c0 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d1ca05f270 con 0x55d1ca153310 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.733+0000 7fd563637740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d1ca0b32b0 con 0x55d1ca1529f0 2024-09-10T10:13:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1620c0 0x55d1ca1644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1620c0 0x55d1ca1644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50508/0 (socket says 172.21.15.45:50508) 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 -- 172.21.15.45:0/4067695628 learned_addr learned my addr 172.21.15.45:0/4067695628 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd55a59c640 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd54b7fe640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd5440a2da0 0x7fd5440a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 -- 172.21.15.45:0/4067695628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 msgr2=0x55d1ca152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 -- 172.21.15.45:0/4067695628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 msgr2=0x55d1ca161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 -- 172.21.15.45:0/4067695628 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ca05e340 con 0x55d1ca1620c0 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd5440a2da0 0x7fd5440a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd54b7fe640 1 -- 172.21.15.45:0/939478503 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd5500cf810 con 0x7fd55c102470 2024-09-10T10:13:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.734+0000 7fd559d9b640 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1620c0 0x55d1ca1644b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd5500cceb0 tx=0x7fd5500911a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7f971afb37e31c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd5440a2da0 0x7fd5440a5260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd554005fd0 tx=0x7fd554004a40 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd548ff9640 1 -- 172.21.15.45:0/4067695628 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55009bae0 con 0x55d1ca1620c0 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd548ff9640 1 -- 172.21.15.45:0/4067695628 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550067850 con 0x55d1ca1620c0 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd548ff9640 1 -- 172.21.15.45:0/4067695628 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55006bca0 con 0x55d1ca1620c0 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd563637740 1 -- 172.21.15.45:0/4067695628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1620c0 msgr2=0x55d1ca1644b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd563637740 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1620c0 0x55d1ca1644b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd5500cceb0 tx=0x7fd5500911a0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd563637740 1 -- 172.21.15.45:0/4067695628 shutdown_connections 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd563637740 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1620c0 0x55d1ca1644b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd563637740 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd563637740 1 --2- 172.21.15.45:0/4067695628 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.735+0000 7fd563637740 1 -- 172.21.15.45:0/4067695628 >> 172.21.15.45:0/4067695628 conn(0x55d1ca14f060 msgr2=0x55d1ca14f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 -- 172.21.15.45:0/4067695628 shutdown_connections 2024-09-10T10:13:36.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 -- 172.21.15.45:0/4067695628 wait complete. 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 -- start start 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 Processor -- start 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1529f0 0x55d1ca15f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca15f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d1ca164cc0 con 0x55d1ca1529f0 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d1ca040290 con 0x55d1ca153310 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55d1ca164e40 con 0x55d1ca1620c0 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd563637740 10 client.0 _ll_register_callbacks cb 0x55d1ca071140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd55a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1529f0 0x55d1ca15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd55a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1529f0 0x55d1ca15f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50514/0 (socket says 172.21.15.45:50514) 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.736+0000 7fd55a59c640 1 -- 172.21.15.45:0/4230353649 learned_addr learned my addr 172.21.15.45:0/4230353649 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd559d9b640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca15f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55a59c640 1 -- 172.21.15.45:0/4230353649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 msgr2=0x55d1ca15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55a59c640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca15fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55a59c640 1 -- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 msgr2=0x55d1ca15f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55a59c640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca15f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55a59c640 1 -- 172.21.15.45:0/4230353649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c9dca210 con 0x55d1ca1529f0 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd559d9b640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd55a59c640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1529f0 0x55d1ca15f000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009930 tx=0x7fd54c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50f24f0f64ab72a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd541ffb640 1 -- 172.21.15.45:0/4230353649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c040070 con 0x55d1ca1529f0 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd541ffb640 1 -- 172.21.15.45:0/4230353649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd54c004240 con 0x55d1ca1529f0 2024-09-10T10:13:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550067850 con 0x55d1ca1529f0 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd541ffb640 1 -- 172.21.15.45:0/4230353649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd54c035040 con 0x55d1ca1529f0 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd563637740 10 client.15807 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.737+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d1c9e96620 con 0x55d1ca1529f0 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd541ffb640 1 -- 172.21.15.45:0/4230353649 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550067850 con 0x55d1ca1529f0 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd541ffb640 1 -- 172.21.15.45:0/4230353649 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd54c004920 con 0x55d1ca1529f0 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd563637740 10 client.15807 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd563637740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd563637740 20 client.15807 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd563637740 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca172090 0x55d1ca174480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd563637740 4 client.15807 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1ca0c4a20 con 0x55d1ca172090 2024-09-10T10:13:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd559d9b640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca172090 0x55d1ca174480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd541ffb640 10 client.15807 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.738+0000 7fd559d9b640 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca172090 0x55d1ca174480 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf69b6ab0d2244c server_cookie=55153a79f510b323 in_seq=0 out_seq=0 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd541ffb640 1 -- 172.21.15.45:0/4230353649 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1ca0c4a20 con 0x55d1ca172090 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd541ffb640 10 client.15807 handle_command_reply: tid=0 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 client.15807 shutdown 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1529f0 msgr2=0x55d1ca15f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1529f0 0x55d1ca15f000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009930 tx=0x7fd54c0317e0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 shutdown_connections 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca172090 0x55d1ca174480 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd541ffb640 0 client.15807 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca153310 0x55d1ca15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 --2- 172.21.15.45:0/4230353649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca1529f0 0x55d1ca15f000 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.739+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 >> 172.21.15.45:0/4230353649 conn(0x55d1ca14f060 msgr2=0x55d1ca154dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.740+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 shutdown_connections 2024-09-10T10:13:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.740+0000 7fd563637740 1 -- 172.21.15.45:0/4230353649 wait complete. 2024-09-10T10:13:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.740+0000 7fd563637740 20 client.15807 trim_cache size 0 max 16384 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.754+0000 7fd563637740 1 Processor -- start 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.754+0000 7fd563637740 1 -- start start 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd563637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1529f0 0x55d1ca179d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd563637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca17a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd563637740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca160860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd563637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1a96d0 con 0x55d1ca153310 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd563637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1abf30 con 0x55d1ca1529f0 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd563637740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1a9d20 con 0x55d1ca1620c0 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca17a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca17a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50524/0 (socket says 172.21.15.45:50524) 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 -- 172.21.15.45:0/268110585 learned_addr learned my addr 172.21.15.45:0/268110585 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd55a59c640 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1529f0 0x55d1ca179d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 -- 172.21.15.45:0/268110585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 msgr2=0x55d1ca160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca160860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 -- 172.21.15.45:0/268110585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1529f0 msgr2=0x55d1ca179d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1529f0 0x55d1ca179d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 -- 172.21.15.45:0/268110585 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ca153140 con 0x55d1ca153310 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd559d9b640 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca17a2d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd550098170 tx=0x7fd550009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b411976b0df435 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd543fff640 1 -- 172.21.15.45:0/268110585 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55009b670 con 0x55d1ca153310 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd543fff640 1 -- 172.21.15.45:0/268110585 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550067850 con 0x55d1ca153310 2024-09-10T10:13:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.755+0000 7fd543fff640 1 -- 172.21.15.45:0/268110585 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55009b850 con 0x55d1ca153310 2024-09-10T10:13:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 -- 172.21.15.45:0/268110585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 msgr2=0x55d1ca17a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca17a2d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd550098170 tx=0x7fd550009310 comp rx=0 tx=0).stop 2024-09-10T10:13:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 -- 172.21.15.45:0/268110585 shutdown_connections 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1620c0 0x55d1ca160860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca17a2d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 --2- 172.21.15.45:0/268110585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1529f0 0x55d1ca179d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 -- 172.21.15.45:0/268110585 >> 172.21.15.45:0/268110585 conn(0x55d1ca14f060 msgr2=0x55d1ca151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 -- 172.21.15.45:0/268110585 shutdown_connections 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 -- 172.21.15.45:0/268110585 wait complete. 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 -- start start 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.756+0000 7fd563637740 1 Processor -- start 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca16a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca165b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1620c0 0x55d1ca166050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1aa3f0 con 0x55d1ca153310 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1ab860 con 0x55d1ca1620c0 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd55c1aaac0 con 0x55d1ca1529f0 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 10 client.0 _ll_register_callbacks cb 0x7fd54c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca165b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50540/0 (socket says 172.21.15.45:50540) 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 -- 172.21.15.45:0/3069166270 learned_addr learned my addr 172.21.15.45:0/3069166270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd55a59c640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca16a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd55ad9d640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1620c0 0x55d1ca166050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 -- 172.21.15.45:0/3069166270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 msgr2=0x55d1ca16a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca16a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 -- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1620c0 msgr2=0x55d1ca166050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1620c0 0x55d1ca166050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 -- 172.21.15.45:0/3069166270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ca166850 con 0x55d1ca153310 2024-09-10T10:13:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd55a59c640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca16a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd559d9b640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca165b10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd5500cd540 tx=0x7fd5500a0240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f9bc21cd762420 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd541ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd541ffb640 1 -- 172.21.15.45:0/3069166270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55009b850 con 0x55d1ca153310 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd541ffb640 1 -- 172.21.15.45:0/3069166270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fd5500cd1c0 con 0x55d1ca153310 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.757+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550067850 con 0x55d1ca153310 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd541ffb640 1 -- 172.21.15.45:0/3069166270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd55009b670 con 0x55d1ca153310 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 10 client.15813 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55d1ca1dcd40 con 0x55d1ca153310 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd541ffb640 1 -- 172.21.15.45:0/3069166270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd550067850 con 0x55d1ca153310 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd541ffb640 1 -- 172.21.15.45:0/3069166270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fd550097a60 con 0x55d1ca153310 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 10 client.15813 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 20 client.15813 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca1dcf60 0x55d1ca1df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 4 client.15813 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:13:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1ca0bed70 con 0x55d1ca1dcf60 2024-09-10T10:13:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.758+0000 7fd55a59c640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca1dcf60 0x55d1ca1df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.759+0000 7fd55a59c640 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca1dcf60 0x55d1ca1df350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8625ebf831285e server_cookie=7137e63454f7250e in_seq=0 out_seq=0 2024-09-10T10:13:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.759+0000 7fd541ffb640 10 client.15813 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.763+0000 7fd541ffb640 1 -- 172.21.15.45:0/3069166270 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1ca0bed70 con 0x55d1ca1dcf60 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.763+0000 7fd541ffb640 10 client.15813 handle_command_reply: tid=0 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.763+0000 7fd563637740 1 client.15813 shutdown 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.763+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 msgr2=0x55d1ca165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca165b10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd5500cd540 tx=0x7fd5500a0240 comp rx=0 tx=0).stop 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 shutdown_connections 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55d1ca1dcf60 0x55d1ca1df350 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d1ca1620c0 0x55d1ca166050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d1ca153310 0x55d1ca165b10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 --2- 172.21.15.45:0/3069166270 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55d1ca1529f0 0x55d1ca16a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd541ffb640 0 client.15813 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 >> 172.21.15.45:0/3069166270 conn(0x55d1ca14f060 msgr2=0x55d1ca15efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 shutdown_connections 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 1 -- 172.21.15.45:0/3069166270 wait complete. 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd563637740 20 client.15813 trim_cache size 0 max 16384 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:13:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:13:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd5440a2da0 msgr2=0x7fd5440a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd5440a2da0 0x7fd5440a5260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd554005fd0 tx=0x7fd554004a40 comp rx=0 tx=0).stop 2024-09-10T10:13:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c102470 msgr2=0x7fd55c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c102470 0x7fd55c1988f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9f0 tx=0x7fd55000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 shutdown_connections 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd55c107b70 0x7fd55c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd55c102470 0x7fd55c1988f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd5440a2da0 0x7fd5440a5260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 --2- 172.21.15.45:0/939478503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd55c101ad0 0x7fd55c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 >> 172.21.15.45:0/939478503 conn(0x7fd55c0fd7d0 msgr2=0x7fd55c0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 shutdown_connections 2024-09-10T10:13:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:36.764+0000 7fd5608a4640 1 -- 172.21.15.45:0/939478503 wait complete. 2024-09-10T10:13:38.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:37 smithi162 ceph-mon[34963]: pgmap v324: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:13:38.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:37 smithi045 ceph-mon[31114]: pgmap v324: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:13:38.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:37 smithi047 ceph-mon[35244]: pgmap v324: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-10T10:13:40.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:39 smithi162 ceph-mon[34963]: pgmap v325: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-10T10:13:40.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:39 smithi045 ceph-mon[31114]: pgmap v325: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-10T10:13:40.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:39 smithi047 ceph-mon[35244]: pgmap v325: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-10T10:13:41.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:41 smithi162 ceph-mon[34963]: pgmap v326: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:41.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:41 smithi045 ceph-mon[31114]: pgmap v326: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:41.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:41 smithi047 ceph-mon[35244]: pgmap v326: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:43.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:43 smithi162 ceph-mon[34963]: pgmap v327: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:43.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:43 smithi045 ceph-mon[31114]: pgmap v327: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:43.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:43 smithi047 ceph-mon[35244]: pgmap v327: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:45.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:45 smithi162 ceph-mon[34963]: pgmap v328: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:45.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:45 smithi045 ceph-mon[31114]: pgmap v328: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:45.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:45 smithi047 ceph-mon[35244]: pgmap v328: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:47.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:47 smithi162 ceph-mon[34963]: pgmap v329: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:47.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:47 smithi045 ceph-mon[31114]: pgmap v329: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:47.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:47 smithi047 ceph-mon[35244]: pgmap v329: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:49.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:49 smithi162 ceph-mon[34963]: pgmap v330: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:49.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:49 smithi045 ceph-mon[31114]: pgmap v330: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:49.409 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:49 smithi047 ceph-mon[35244]: pgmap v330: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 -- 172.21.15.45:0/3002389885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0101ad0 msgr2=0x7f5ab0101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 --2- 172.21.15.45:0/3002389885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0101ad0 0x7f5ab0101f30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009930 tx=0x7f5aa002a420 comp rx=0 tx=0).stop 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 -- 172.21.15.45:0/3002389885 shutdown_connections 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 --2- 172.21.15.45:0/3002389885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ab0102470 0x7f5ab010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 --2- 172.21.15.45:0/3002389885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0101ad0 0x7f5ab0101f30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 --2- 172.21.15.45:0/3002389885 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab0107b70 0x7f5ab0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 -- 172.21.15.45:0/3002389885 >> 172.21.15.45:0/3002389885 conn(0x7f5ab00fd7d0 msgr2=0x7f5ab00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.865+0000 7f5ab725e640 1 -- 172.21.15.45:0/3002389885 shutdown_connections 2024-09-10T10:13:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 -- 172.21.15.45:0/3002389885 wait complete. 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 Processor -- start 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 -- start start 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab0101ad0 0x7f5ab01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ab0102470 0x7f5ab01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0107b70 0x7f5ab019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0111f20 con 0x7f5ab0107b70 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0111da0 con 0x7f5ab0102470 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.866+0000 7f5ab725e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01120a0 con 0x7f5ab0101ad0 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0107b70 0x7f5ab019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0107b70 0x7f5ab019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39204/0 (socket says 172.21.15.45:39204) 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 -- 172.21.15.45:0/2311667213 learned_addr learned my addr 172.21.15.45:0/2311667213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5aaffff640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ab0102470 0x7f5ab01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab0101ad0 0x7f5ab01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 -- 172.21.15.45:0/2311667213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab0101ad0 msgr2=0x7f5ab01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab0101ad0 0x7f5ab01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 -- 172.21.15.45:0/2311667213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ab0102470 msgr2=0x7f5ab01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ab0102470 0x7f5ab01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 -- 172.21.15.45:0/2311667213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab019d3a0 con 0x7f5ab0107b70 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab0101ad0 0x7f5ab01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0107b70 0x7f5ab019cc70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800eaa0 tx=0x7f5aa800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5aadffb640 1 -- 172.21.15.45:0/2311667213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800cdb0 con 0x7f5ab0107b70 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5aadffb640 1 -- 172.21.15.45:0/2311667213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8005020 con 0x7f5ab0107b70 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5aadffb640 1 -- 172.21.15.45:0/2311667213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80116b0 con 0x7f5ab0107b70 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.867+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab019d690 con 0x7f5ab0107b70 2024-09-10T10:13:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.868+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab01a4f20 con 0x7f5ab0107b70 2024-09-10T10:13:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.870+0000 7f5aadffb640 1 -- 172.21.15.45:0/2311667213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5aa8005560 con 0x7f5ab0107b70 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 Processor -- start 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 -- start start 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5aadffb640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5a9c0a2e20 0x7f5a9c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d479d0 0x5650b8d47db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d570c0 0x5650b8d594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650b8cac2f0 con 0x5650b8d482f0 2024-09-10T10:13:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5650b8c54270 con 0x5650b8d570c0 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5650b8ca82b0 con 0x5650b8d479d0 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5aaffff640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5a9c0a2e20 0x7f5a9c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.871+0000 7f5aadffb640 1 -- 172.21.15.45:0/2311667213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5aa80cf6c0 con 0x7f5ab0107b70 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d56b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d570c0 0x5650b8d594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d56b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39216/0 (socket says 172.21.15.45:39216) 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d570c0 0x5650b8d594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:48834/0 (socket says 172.21.15.45:48834) 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5aaffff640 1 -- 172.21.15.45:0/653070876 learned_addr learned my addr 172.21.15.45:0/653070876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d479d0 0x5650b8d47db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 -- 172.21.15.45:0/653070876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d479d0 msgr2=0x5650b8d47db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d479d0 0x5650b8d47db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 -- 172.21.15.45:0/653070876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 msgr2=0x5650b8d56b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d56b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 -- 172.21.15.45:0/653070876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b8c53340 con 0x5650b8d570c0 2024-09-10T10:13:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d479d0 0x5650b8d47db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:13:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5aaffff640 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:13:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5aaffff640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5a9c0a2e20 0x7f5a9c0a52e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0032fd0 tx=0x7f5aa0005fd0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d570c0 0x5650b8d594b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80ccd90 tx=0x7f5aa809bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de67de3670bf93e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5aa6ffd640 1 -- 172.21.15.45:0/653070876 <== mon.2 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa801f040 con 0x5650b8d570c0 2024-09-10T10:13:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.872+0000 7f5aa6ffd640 1 -- 172.21.15.45:0/653070876 <== mon.2 v2:172.21.15.47:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8099e60 con 0x5650b8d570c0 2024-09-10T10:13:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5aa6ffd640 1 -- 172.21.15.45:0/653070876 <== mon.2 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80762d0 con 0x5650b8d570c0 2024-09-10T10:13:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/653070876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d570c0 msgr2=0x5650b8d594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d570c0 0x5650b8d594b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80ccd90 tx=0x7f5aa809bca0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/653070876 shutdown_connections 2024-09-10T10:13:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d570c0 0x5650b8d594b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/653070876 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d479d0 0x5650b8d47db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/653070876 >> 172.21.15.45:0/653070876 conn(0x5650b8d44060 msgr2=0x5650b8d44440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/653070876 shutdown_connections 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.873+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/653070876 wait complete. 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 -- start start 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 Processor -- start 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d54040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d570c0 0x5650b8d54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5650b8d59cf0 con 0x5650b8d482f0 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5650b8c35290 con 0x5650b8d479d0 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x5650b8d59e70 con 0x5650b8d570c0 2024-09-10T10:13:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5ab9ffa740 10 client.0 _ll_register_callbacks cb 0x5650b8c66140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.874+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d54580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39218/0 (socket says 172.21.15.45:39218) 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 -- 172.21.15.45:0/1880604459 learned_addr learned my addr 172.21.15.45:0/1880604459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d54040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d570c0 0x5650b8d54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 -- 172.21.15.45:0/1880604459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d570c0 msgr2=0x5650b8d54ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d570c0 0x5650b8d54ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 -- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 msgr2=0x5650b8d54040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d54040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 -- 172.21.15.45:0/1880604459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b89bf210 con 0x5650b8d482f0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5aaffff640 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d54580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0005780 tx=0x7f5aa00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab23779e9eae815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5a8ffff640 1 -- 172.21.15.45:0/1880604459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0003ac0 con 0x5650b8d482f0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5a8ffff640 1 -- 172.21.15.45:0/1880604459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5aa0003c60 con 0x5650b8d482f0 2024-09-10T10:13:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.875+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa8099e60 con 0x5650b8d482f0 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5a8ffff640 1 -- 172.21.15.45:0/1880604459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5aa005b050 con 0x5650b8d482f0 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 10 client.15825 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5650b8a8b620 con 0x5650b8d482f0 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5a8ffff640 1 -- 172.21.15.45:0/1880604459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8099e60 con 0x5650b8d482f0 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5a8ffff640 1 -- 172.21.15.45:0/1880604459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5aa003d450 con 0x5650b8d482f0 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 10 client.15825 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 20 client.15825 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8d67150 0x5650b8d69540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 4 client.15825 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b8cb9a20 con 0x5650b8d67150 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.876+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8d67150 0x5650b8d69540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.877+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8d67150 0x5650b8d69540 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13cbf78918d6972 server_cookie=cdec7c1089dbb1fc in_seq=0 out_seq=0 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.877+0000 7f5a8ffff640 10 client.15825 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5a8ffff640 1 -- 172.21.15.45:0/1880604459 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650b8cb9a20 con 0x5650b8d67150 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5a8ffff640 10 client.15825 handle_command_reply: tid=0 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 client.15825 shutdown 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 msgr2=0x5650b8d54580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d54580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0005780 tx=0x7f5aa00058d0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 shutdown_connections 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8d67150 0x5650b8d69540 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d570c0 0x5650b8d54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d482f0 0x5650b8d54580 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/1880604459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d54040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5a8ffff640 0 client.15825 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 >> 172.21.15.45:0/1880604459 conn(0x5650b8d44060 msgr2=0x5650b8d49dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 shutdown_connections 2024-09-10T10:13:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/1880604459 wait complete. 2024-09-10T10:13:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.878+0000 7f5ab9ffa740 20 client.15825 trim_cache size 0 max 16384 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 Processor -- start 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 -- start start 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d430b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d6ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d6f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01a9580 con 0x5650b8d570c0 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01abde0 con 0x5650b8d479d0 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01a9bd0 con 0x5650b8d482f0 2024-09-10T10:13:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d6f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d6f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39224/0 (socket says 172.21.15.45:39224) 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3549576828 learned_addr learned my addr 172.21.15.45:0/3549576828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5aaffff640 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d6ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3549576828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 msgr2=0x5650b8d6ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d6ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3549576828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 msgr2=0x5650b8d430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3549576828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b8d48120 con 0x5650b8d570c0 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.893+0000 7f5aaffff640 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d6ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d6f320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80a00d0 tx=0x7f5aa8099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b874d6091b71a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5aa67fc640 1 -- 172.21.15.45:0/3549576828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa801f040 con 0x5650b8d570c0 2024-09-10T10:13:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5aa67fc640 1 -- 172.21.15.45:0/3549576828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa80051d0 con 0x5650b8d570c0 2024-09-10T10:13:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5aa67fc640 1 -- 172.21.15.45:0/3549576828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80978b0 con 0x5650b8d570c0 2024-09-10T10:13:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3549576828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 msgr2=0x5650b8d6f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d6f320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80a00d0 tx=0x7f5aa8099d90 comp rx=0 tx=0).stop 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3549576828 shutdown_connections 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d6f320 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d6ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3549576828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3549576828 >> 172.21.15.45:0/3549576828 conn(0x5650b8d44060 msgr2=0x5650b8d429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3549576828 shutdown_connections 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.894+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3549576828 wait complete. 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 -- start start 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 Processor -- start 2024-09-10T10:13:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d5b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01aa2a0 con 0x5650b8d570c0 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01ab710 con 0x5650b8d479d0 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01aa970 con 0x5650b8d482f0 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab9ffa740 10 client.0 _ll_register_callbacks cb 0x7f5aa0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab57fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d5bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39240/0 (socket says 172.21.15.45:39240) 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3671553180 learned_addr learned my addr 172.21.15.45:0/3671553180 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.895+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d5b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5aaffff640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3671553180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 msgr2=0x5650b8d5b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d5b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 msgr2=0x5650b8d5b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d5b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab57fb640 1 -- 172.21.15.45:0/3671553180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b8d63ad0 con 0x5650b8d570c0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5aaffff640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d5b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab57fb640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d5bbe0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5aa809b810 tx=0x7f5aa8098400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e323e0258415b677 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5a8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5a8ffff640 1 -- 172.21.15.45:0/3671553180 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa801f050 con 0x5650b8d570c0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5a8ffff640 1 -- 172.21.15.45:0/3671553180 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7f5aa80ba2b0 con 0x5650b8d570c0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa80051d0 con 0x5650b8d570c0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5a8ffff640 1 -- 172.21.15.45:0/3671553180 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5aa8097e00 con 0x5650b8d570c0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab9ffa740 10 client.15831 fetch_fsmap learned FSMap version 21 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5650b8dd2190 con 0x5650b8d570c0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5a8ffff640 1 -- 172.21.15.45:0/3671553180 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa80051d0 con 0x5650b8d570c0 2024-09-10T10:13:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.896+0000 7f5a8ffff640 1 -- 172.21.15.45:0/3671553180 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f5aa800f040 con 0x5650b8d570c0 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab9ffa740 10 client.15831 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab9ffa740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab9ffa740 20 client.15831 populate_metadata read hostname 'smithi045' 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8dd2410 0x5650b8dd4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab9ffa740 4 client.15831 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650b8cb3d70 con 0x5650b8dd2410 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8dd2410 0x5650b8dd4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5a8ffff640 10 client.15831 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.897+0000 7f5ab4ffa640 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8dd2410 0x5650b8dd4800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9747f7668e695dd9 server_cookie=8fcafb726f5aa5a3 in_seq=0 out_seq=0 2024-09-10T10:13:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5a8ffff640 1 -- 172.21.15.45:0/3671553180 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650b8cb3d70 con 0x5650b8dd2410 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5a8ffff640 10 client.15831 handle_command_reply: tid=0 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 client.15831 shutdown 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 msgr2=0x5650b8d5bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d5bbe0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5aa809b810 tx=0x7f5aa8098400 comp rx=0 tx=0).stop 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 shutdown_connections 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x5650b8dd2410 0x5650b8dd4800 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5650b8d570c0 0x5650b8d5bbe0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x5650b8d482f0 0x5650b8d5b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5a8ffff640 0 client.15831 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 --2- 172.21.15.45:0/3671553180 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5650b8d479d0 0x5650b8d5b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 >> 172.21.15.45:0/3671553180 conn(0x5650b8d44060 msgr2=0x5650b8d431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.902+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 shutdown_connections 2024-09-10T10:13:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab9ffa740 1 -- 172.21.15.45:0/3671553180 wait complete. 2024-09-10T10:13:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab9ffa740 20 client.15831 trim_cache size 0 max 16384 2024-09-10T10:13:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:13:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:13:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:13:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5a9c0a2e20 msgr2=0x7f5a9c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5a9c0a2e20 0x7f5a9c0a52e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0032fd0 tx=0x7f5aa0005fd0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0107b70 msgr2=0x7f5ab019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0107b70 0x7f5ab019cc70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800eaa0 tx=0x7f5aa800ef60 comp rx=0 tx=0).stop 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 shutdown_connections 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ab0107b70 0x7f5ab019cc70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5a9c0a2e20 0x7f5a9c0a52e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ab0102470 0x7f5ab01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 --2- 172.21.15.45:0/2311667213 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5ab0101ad0 0x7f5ab01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 >> 172.21.15.45:0/2311667213 conn(0x7f5ab00fd7d0 msgr2=0x7f5ab00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:13:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 shutdown_connections 2024-09-10T10:13:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:13:49.903+0000 7f5ab725e640 1 -- 172.21.15.45:0/2311667213 wait complete. 2024-09-10T10:13:51.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:51 smithi162 ceph-mon[34963]: pgmap v331: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s 2024-09-10T10:13:51.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:51 smithi045 ceph-mon[31114]: pgmap v331: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s 2024-09-10T10:13:51.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:51 smithi047 ceph-mon[35244]: pgmap v331: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 2 op/s 2024-09-10T10:13:53.400 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:53 smithi162 ceph-mon[34963]: pgmap v332: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:13:53.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:53 smithi045 ceph-mon[31114]: pgmap v332: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:13:53.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:53 smithi047 ceph-mon[35244]: pgmap v332: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-10T10:13:55.399 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:55 smithi162 ceph-mon[34963]: pgmap v333: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:55.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:55 smithi045 ceph-mon[31114]: pgmap v333: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:55.410 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:55 smithi047 ceph-mon[35244]: pgmap v333: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:57.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:57 smithi162 ceph-mon[34963]: pgmap v334: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:57.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:57 smithi045 ceph-mon[31114]: pgmap v334: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:57.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:57 smithi047 ceph-mon[35244]: pgmap v334: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:59.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:13:59 smithi162 ceph-mon[34963]: pgmap v335: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:59.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:13:59 smithi045 ceph-mon[31114]: pgmap v335: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:13:59.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:13:59 smithi047 ceph-mon[35244]: pgmap v335: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:14:01.579 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:01 smithi045 ceph-mon[31114]: pgmap v336: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:14:01.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:01 smithi162 ceph-mon[34963]: pgmap v336: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:14:01.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:01 smithi047 ceph-mon[35244]: pgmap v336: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.005+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3066448741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c1009f0 msgr2=0x7fa99c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.005+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3066448741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c1009f0 0x7fa99c100e50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009930 tx=0x7fa98c02a420 comp rx=0 tx=0).stop 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3066448741 shutdown_connections 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3066448741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa99c101390 0x7fa99c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3066448741 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c1009f0 0x7fa99c100e50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3066448741 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa99c106a00 0x7fa99c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3066448741 >> 172.21.15.45:0/3066448741 conn(0x7fa99c0fc6b0 msgr2=0x7fa99c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3066448741 shutdown_connections 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3066448741 wait complete. 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 Processor -- start 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 -- start start 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.006+0000 7fa9a3cdd640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa99c1009f0 0x7fa99c0fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a3cdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c101390 0x7fa99c0fd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a3cdd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa99c106a00 0x7fa99c0fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a3cdd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c110e40 con 0x7fa99c101390 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a3cdd640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c110cc0 con 0x7fa99c1009f0 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a3cdd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c110fc0 con 0x7fa99c106a00 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c101390 0x7fa99c0fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c101390 0x7fa99c0fd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33178/0 (socket says 172.21.15.45:33178) 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 -- 172.21.15.45:0/3043168679 learned_addr learned my addr 172.21.15.45:0/3043168679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa99c106a00 0x7fa99c0fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa99c1009f0 0x7fa99c0fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 -- 172.21.15.45:0/3043168679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa99c106a00 msgr2=0x7fa99c0fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa99c106a00 0x7fa99c0fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 -- 172.21.15.45:0/3043168679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa99c1009f0 msgr2=0x7fa99c0fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa99c1009f0 0x7fa99c0fcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 -- 172.21.15.45:0/3043168679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c1a68e0 con 0x7fa99c101390 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa99c106a00 0x7fa99c0fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa9a1278640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c101390 0x7fa99c0fd190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009230 tx=0x7fa98c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.007+0000 7fa992ffd640 1 -- 172.21.15.45:0/3043168679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c03f030 con 0x7fa99c101390 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.008+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c1a6b70 con 0x7fa99c101390 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.008+0000 7fa992ffd640 1 -- 172.21.15.45:0/3043168679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98c004440 con 0x7fa99c101390 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.008+0000 7fa992ffd640 1 -- 172.21.15.45:0/3043168679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c035070 con 0x7fa99c101390 2024-09-10T10:14:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.008+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c1a70b0 con 0x7fa99c101390 2024-09-10T10:14:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.010+0000 7fa992ffd640 1 -- 172.21.15.45:0/3043168679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa98c042030 con 0x7fa99c101390 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.010+0000 7fa9a6a74740 1 Processor -- start 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a6a74740 1 -- start start 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d1f10b2f0 con 0x561d1f1b1cb0 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561d1f0b3270 con 0x561d1f1a6af0 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561d1f1072b0 con 0x561d1f1a7410 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa992ffd640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa9880a2da0 0x7fa9880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a1278640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a1278640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33180/0 (socket says 172.21.15.45:33180) 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a1278640 1 -- 172.21.15.45:0/3333534454 learned_addr learned my addr 172.21.15.45:0/3333534454 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa9880a2da0 0x7fa9880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.011+0000 7fa992ffd640 1 -- 172.21.15.45:0/3043168679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa98c0ef750 con 0x7fa99c101390 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a1a79640 1 -- 172.21.15.45:0/3333534454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 msgr2=0x561d1f1a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1a6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a1a79640 1 -- 172.21.15.45:0/3333534454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 msgr2=0x561d1f1b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a1a79640 1 -- 172.21.15.45:0/3333534454 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1f0b2340 con 0x561d1f1a7410 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a1278640 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa9880a2da0 0x7fa9880a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa994005fd0 tx=0x7fa994011810 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1b1770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa998009c00 tx=0x7fa998011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1dc6383b0dab2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa987fff640 1 -- 172.21.15.45:0/3333534454 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998009db0 con 0x561d1f1a7410 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa987fff640 1 -- 172.21.15.45:0/3333534454 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa998005020 con 0x561d1f1a7410 2024-09-10T10:14:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.012+0000 7fa987fff640 1 -- 172.21.15.45:0/3333534454 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998004b60 con 0x561d1f1a7410 2024-09-10T10:14:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3333534454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 msgr2=0x561d1f1b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1b1770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa998009c00 tx=0x7fa998011810 comp rx=0 tx=0).stop 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3333534454 shutdown_connections 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1b1770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3333534454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1a6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3333534454 >> 172.21.15.45:0/3333534454 conn(0x561d1f1a3060 msgr2=0x561d1f1a3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3333534454 shutdown_connections 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3333534454 wait complete. 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.013+0000 7fa9a6a74740 1 -- start start 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 1 Processor -- start 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d1f1b4880 con 0x561d1f1b1cb0 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561d1f094290 con 0x561d1f1a6af0 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x561d1f1b4a00 con 0x561d1f1a7410 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a6a74740 10 client.0 _ll_register_callbacks cb 0x561d1f0c5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33184/0 (socket says 172.21.15.45:33184) 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 -- 172.21.15.45:0/3644031167 learned_addr learned my addr 172.21.15.45:0/3644031167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a1278640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 -- 172.21.15.45:0/3644031167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 msgr2=0x561d1f1af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 -- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 msgr2=0x561d1f1aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a227a640 1 -- 172.21.15.45:0/3644031167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1ee1e210 con 0x561d1f1b1cb0 2024-09-10T10:14:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.014+0000 7fa9a1278640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1af570 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa994003ea0 tx=0x7fa994013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9215f79f783c1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9857fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9857fa640 1 -- 172.21.15.45:0/3644031167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994011020 con 0x561d1f1b1cb0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9857fa640 1 -- 172.21.15.45:0/3644031167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa9940111c0 con 0x561d1f1b1cb0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa998005020 con 0x561d1f1b1cb0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9857fa640 1 -- 172.21.15.45:0/3644031167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa994004b20 con 0x561d1f1b1cb0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9a6a74740 10 client.15843 fetch_fsmap learned FSMap version 21 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561d1eeea620 con 0x561d1f1b1cb0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.015+0000 7fa9857fa640 1 -- 172.21.15.45:0/3644031167 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa998005020 con 0x561d1f1b1cb0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9857fa640 1 -- 172.21.15.45:0/3644031167 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa994010690 con 0x561d1f1b1cb0 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a6a74740 10 client.15843 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a6a74740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a6a74740 20 client.15843 populate_metadata read hostname 'smithi045' 2024-09-10T10:14:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f1c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a6a74740 4 client.15843 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d1f118a20 con 0x561d1f1c1b50 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f1c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9857fa640 10 client.15843 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.016+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f1c3fc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240724a511d1aef3 server_cookie=320301fdfee44405 in_seq=0 out_seq=0 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9857fa640 1 -- 172.21.15.45:0/3644031167 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d1f118a20 con 0x561d1f1c1b50 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9857fa640 10 client.15843 handle_command_reply: tid=0 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 client.15843 shutdown 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 msgr2=0x561d1f1af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1af570 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa994003ea0 tx=0x7fa994013070 comp rx=0 tx=0).stop 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 shutdown_connections 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f1c3fc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1b1cb0 0x561d1f1af570 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9857fa640 0 client.15843 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1a7410 0x561d1f1af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3644031167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a6af0 0x561d1f1aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 >> 172.21.15.45:0/3644031167 conn(0x561d1f1a3060 msgr2=0x561d1f1a8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 shutdown_connections 2024-09-10T10:14:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.017+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3644031167 wait complete. 2024-09-10T10:14:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.018+0000 7fa9a6a74740 20 client.15843 trim_cache size 0 max 16384 2024-09-10T10:14:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 Processor -- start 2024-09-10T10:14:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 -- start start 2024-09-10T10:14:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c1ada50 con 0x561d1f1a6af0 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c1b02b0 con 0x561d1f1a7410 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c1ae0a0 con 0x561d1f1b1cb0 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.032+0000 7fa9a1a79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33198/0 (socket says 172.21.15.45:33198) 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 -- 172.21.15.45:0/3488147321 learned_addr learned my addr 172.21.15.45:0/3488147321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1278640 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a227a640 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 -- 172.21.15.45:0/3488147321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 msgr2=0x561d1f1c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 -- 172.21.15.45:0/3488147321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 msgr2=0x561d1f1af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 -- 172.21.15.45:0/3488147321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d1f1a2e90 con 0x561d1f1a6af0 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1a9560 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa99800a010 tx=0x7fa998011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46917b9c584fde13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9877fe640 1 -- 172.21.15.45:0/3488147321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99802d070 con 0x561d1f1a6af0 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9877fe640 1 -- 172.21.15.45:0/3488147321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa998005350 con 0x561d1f1a6af0 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.033+0000 7fa9877fe640 1 -- 172.21.15.45:0/3488147321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998004c10 con 0x561d1f1a6af0 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3488147321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 msgr2=0x561d1f1a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1a9560 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa99800a010 tx=0x7fa998011810 comp rx=0 tx=0).stop 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3488147321 shutdown_connections 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/3488147321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1a9560 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3488147321 >> 172.21.15.45:0/3488147321 conn(0x561d1f1a3060 msgr2=0x561d1f1b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3488147321 shutdown_connections 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 -- 172.21.15.45:0/3488147321 wait complete. 2024-09-10T10:14:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 -- start start 2024-09-10T10:14:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.034+0000 7fa9a6a74740 1 Processor -- start 2024-09-10T10:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c1ae770 con 0x561d1f1a6af0 2024-09-10T10:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c1afbe0 con 0x561d1f1a7410 2024-09-10T10:14:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c1aee40 con 0x561d1f1b1cb0 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 10 client.0 _ll_register_callbacks cb 0x7fa994003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33200/0 (socket says 172.21.15.45:33200) 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 -- 172.21.15.45:0/29779509 learned_addr learned my addr 172.21.15.45:0/29779509 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1278640 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a227a640 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 -- 172.21.15.45:0/29779509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 msgr2=0x561d1f1b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 -- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 msgr2=0x561d1f1b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 -- 172.21.15.45:0/29779509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1f22b8f0 con 0x561d1f1a6af0 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a1a79640 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1ba580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa998009990 tx=0x7fa9980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8214b9e63520bf75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9857fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9857fa640 1 -- 172.21.15.45:0/29779509 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa998035030 con 0x561d1f1a6af0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9857fa640 1 -- 172.21.15.45:0/29779509 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8556+0+0 (secure 0 0 0) 0x7fa998016040 con 0x561d1f1a6af0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.035+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa998005350 con 0x561d1f1a6af0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9857fa640 1 -- 172.21.15.45:0/29779509 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa99802d070 con 0x561d1f1a6af0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 10 client.15849 fetch_fsmap learned FSMap version 21 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561d1f22caa0 con 0x561d1f1a6af0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9857fa640 1 -- 172.21.15.45:0/29779509 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa998005350 con 0x561d1f1a6af0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9857fa640 1 -- 172.21.15.45:0/29779509 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fa998019b10 con 0x561d1f1a6af0 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 10 client.15849 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 20 client.15849 populate_metadata read hostname 'smithi045' 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f22ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 4 client.15849 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:14:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d1f112d70 con 0x561d1f1c1b50 2024-09-10T10:14:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.036+0000 7fa9a1278640 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f22ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.037+0000 7fa9a1278640 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f22ef70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ccbc7711a0be01 server_cookie=9cf70a7749c69998 in_seq=0 out_seq=0 2024-09-10T10:14:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:03.037+0000 7fa9857fa640 10 client.15849 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:03.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:03 smithi045 ceph-mon[31114]: pgmap v337: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:14:03.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:03 smithi162 ceph-mon[34963]: pgmap v337: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:14:03.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:03 smithi047 ceph-mon[35244]: pgmap v337: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9857fa640 1 -- 172.21.15.45:0/29779509 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d1f112d70 con 0x561d1f1c1b50 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9857fa640 10 client.15849 handle_command_reply: tid=0 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 client.15849 shutdown 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 msgr2=0x561d1f1ba580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1ba580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa998009990 tx=0x7fa9980046c0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 shutdown_connections 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x561d1f1c1b50 0x561d1f22ef70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x561d1f1b1cb0 0x561d1f1b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561d1f1a7410 0x561d1f1b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 --2- 172.21.15.45:0/29779509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d1f1a6af0 0x561d1f1ba580 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 >> 172.21.15.45:0/29779509 conn(0x561d1f1a3060 msgr2=0x561d1f1b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9857fa640 0 client.15849 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 shutdown_connections 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 1 -- 172.21.15.45:0/29779509 wait complete. 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.423+0000 7fa9a6a74740 20 client.15849 trim_cache size 0 max 16384 2024-09-10T10:14:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:14:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:14:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:14:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa9880a2da0 msgr2=0x7fa9880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa9880a2da0 0x7fa9880a5260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa994005fd0 tx=0x7fa994011810 comp rx=0 tx=0).stop 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c101390 msgr2=0x7fa99c0fd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c101390 0x7fa99c0fd190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009230 tx=0x7fa98c004040 comp rx=0 tx=0).stop 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 shutdown_connections 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa99c106a00 0x7fa99c0fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c101390 0x7fa99c0fd190 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fa9880a2da0 0x7fa9880a5260 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 --2- 172.21.15.45:0/3043168679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa99c1009f0 0x7fa99c0fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 >> 172.21.15.45:0/3043168679 conn(0x7fa99c0fc6b0 msgr2=0x7fa99c06a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 shutdown_connections 2024-09-10T10:14:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:04.424+0000 7fa9a3cdd640 1 -- 172.21.15.45:0/3043168679 wait complete. 2024-09-10T10:14:05.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:05 smithi162 ceph-mon[34963]: pgmap v338: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-10T10:14:05.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:05 smithi045 ceph-mon[31114]: pgmap v338: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-10T10:14:05.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:05 smithi047 ceph-mon[35244]: pgmap v338: 97 pgs: 97 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-10T10:14:06.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:06 smithi162 ceph-mon[34963]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T10:14:06.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:06 smithi045 ceph-mon[31114]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T10:14:06.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:06 smithi047 ceph-mon[35244]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T10:14:07.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:07 smithi162 ceph-mon[34963]: pgmap v340: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-10T10:14:07.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:07 smithi045 ceph-mon[31114]: pgmap v340: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-10T10:14:07.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:07 smithi047 ceph-mon[35244]: pgmap v340: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 31 GiB data, 97 GiB used, 975 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-10T10:14:09.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:09 smithi162 ceph-mon[34963]: pgmap v341: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 26 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:09.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:09 smithi045 ceph-mon[31114]: pgmap v341: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 26 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:09.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:09 smithi047 ceph-mon[35244]: pgmap v341: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 26 GiB data, 86 GiB used, 987 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:11.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:11 smithi162 ceph-mon[34963]: pgmap v342: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:11.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:11 smithi045 ceph-mon[31114]: pgmap v342: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:11.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:11 smithi047 ceph-mon[35244]: pgmap v342: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 23 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:13.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:13 smithi162 ceph-mon[34963]: pgmap v343: 97 pgs: 97 active+clean; 21 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:13.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:13 smithi045 ceph-mon[31114]: pgmap v343: 97 pgs: 97 active+clean; 21 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:13.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:13 smithi047 ceph-mon[35244]: pgmap v343: 97 pgs: 97 active+clean; 21 GiB data, 76 GiB used, 996 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2024-09-10T10:14:15.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:15 smithi162 ceph-mon[34963]: pgmap v344: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-10T10:14:15.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:15 smithi045 ceph-mon[31114]: pgmap v344: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-10T10:14:15.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:15 smithi047 ceph-mon[35244]: pgmap v344: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-10T10:14:16.715 INFO:tasks.workunit.client.0.smithi045.stdout:stopping iogen 2024-09-10T10:14:16.715 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo 'stopping iogen' 2024-09-10T10:14:16.715 INFO:tasks.workunit.client.0.smithi045.stderr:+ ./iogen -k 2024-09-10T10:14:16.716 INFO:tasks.workunit.client.0.smithi045.stdout:OK 2024-09-10T10:14:16.716 INFO:tasks.workunit.client.0.smithi045.stderr:+ echo OK 2024-09-10T10:14:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 -- 172.21.15.45:0/819775916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 msgr2=0x7f9b94106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/819775916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b94106da0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9b84009930 tx=0x7f9b8402a420 comp rx=0 tx=0).stop 2024-09-10T10:14:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 -- 172.21.15.45:0/819775916 shutdown_connections 2024-09-10T10:14:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/819775916 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b94101350 0x7f9b9410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/819775916 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b941009b0 0x7f9b94100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/819775916 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b94106da0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.529 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 -- 172.21.15.45:0/819775916 >> 172.21.15.45:0/819775916 conn(0x7f9b94077990 msgr2=0x7f9b940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 -- 172.21.15.45:0/819775916 shutdown_connections 2024-09-10T10:14:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.526+0000 7f9b9a34b640 1 -- 172.21.15.45:0/819775916 wait complete. 2024-09-10T10:14:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 Processor -- start 2024-09-10T10:14:17.530 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 -- start start 2024-09-10T10:14:17.531 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b941009b0 0x7f9b94198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b94101350 0x7f9b94198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b9419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94111eb0 con 0x7f9b941069c0 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94111d30 con 0x7f9b94101350 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b9a34b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94112030 con 0x7f9b941009b0 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b988e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b9419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b988e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b9419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37348/0 (socket says 172.21.15.45:37348) 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b988e8640 1 -- 172.21.15.45:0/2918176630 learned_addr learned my addr 172.21.15.45:0/2918176630 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.527+0000 7f9b937fe640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b94101350 0x7f9b94198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b93fff640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b941009b0 0x7f9b94198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b988e8640 1 -- 172.21.15.45:0/2918176630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b941009b0 msgr2=0x7f9b94198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b988e8640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b941009b0 0x7f9b94198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b988e8640 1 -- 172.21.15.45:0/2918176630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b94101350 msgr2=0x7f9b94198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b988e8640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b94101350 0x7f9b94198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b988e8640 1 -- 172.21.15.45:0/2918176630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9419d3f0 con 0x7f9b941069c0 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b988e8640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b9419ccc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00eb10 tx=0x7f9b8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b917fa640 1 -- 172.21.15.45:0/2918176630 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c004b20 con 0x7f9b941069c0 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9419d6e0 con 0x7f9b941069c0 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b917fa640 1 -- 172.21.15.45:0/2918176630 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b8c004cc0 con 0x7f9b941069c0 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b917fa640 1 -- 172.21.15.45:0/2918176630 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c010760 con 0x7f9b941069c0 2024-09-10T10:14:17.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.528+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b941a4f70 con 0x7f9b941069c0 2024-09-10T10:14:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b917fa640 1 -- 172.21.15.45:0/2918176630 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9b8c005020 con 0x7f9b941069c0 2024-09-10T10:14:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 Processor -- start 2024-09-10T10:14:17.538 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 -- start start 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652757af0 0x55c652757ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652758410 0x55c652762770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c6527650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c6526bc2f0 con 0x55c652758410 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c652664270 con 0x55c652757af0 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c6526b82b0 con 0x55c652762cb0 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.531+0000 7f9b917fa640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f9b7c0a2da0 0x7f9b7c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652758410 0x55c652762770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652758410 0x55c652762770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37362/0 (socket says 172.21.15.45:37362) 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 -- 172.21.15.45:0/1039872803 learned_addr learned my addr 172.21.15.45:0/1039872803 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b988e8640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f9b7c0a2da0 0x7f9b7c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.539 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b937fe640 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c6527650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b917fa640 1 -- 172.21.15.45:0/2918176630 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f9b8c0cf960 con 0x7f9b941069c0 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b937fe640 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652757af0 0x55c652757ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 -- 172.21.15.45:0/1039872803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 msgr2=0x55c6527650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c6527650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 -- 172.21.15.45:0/1039872803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652757af0 msgr2=0x55c652757ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652757af0 0x55c652757ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 -- 172.21.15.45:0/1039872803 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c652663340 con 0x55c652758410 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b937fe640 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652757af0 0x55c652757ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b93fff640 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652758410 0x55c652762770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b8402a940 tx=0x7f9b84005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519ae683dabd8876 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b988e8640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f9b7c0a2da0 0x7f9b7c0a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c09bb60 tx=0x7f9b8c01bb30 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b7a7fc640 1 -- 172.21.15.45:0/1039872803 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84040070 con 0x55c652758410 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b7a7fc640 1 -- 172.21.15.45:0/1039872803 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b84031870 con 0x55c652758410 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.532+0000 7f9b7a7fc640 1 -- 172.21.15.45:0/1039872803 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84035040 con 0x55c652758410 2024-09-10T10:14:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1039872803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652758410 msgr2=0x55c652762770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652758410 0x55c652762770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b8402a940 tx=0x7f9b84005bd0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1039872803 shutdown_connections 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c6527650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652758410 0x55c652762770 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1039872803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652757af0 0x55c652757ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1039872803 >> 172.21.15.45:0/1039872803 conn(0x55c652754060 msgr2=0x55c652754440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1039872803 shutdown_connections 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.533+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1039872803 wait complete. 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 -- start start 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 Processor -- start 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.541 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c652760600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c652765880 con 0x55c652757af0 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c652645290 con 0x55c652758410 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x55c652765a00 con 0x55c652762cb0 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b9d1c4740 10 client.0 _ll_register_callbacks cb 0x55c652676140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37374/0 (socket says 172.21.15.45:37374) 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b937fe640 1 -- 172.21.15.45:0/3906348742 learned_addr learned my addr 172.21.15.45:0/3906348742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.534+0000 7f9b93fff640 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b937fe640 1 -- 172.21.15.45:0/3906348742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 msgr2=0x55c652760600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b937fe640 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c652760600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b937fe640 1 -- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 msgr2=0x55c6527600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b937fe640 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b937fe640 1 -- 172.21.15.45:0/3906348742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6523cf210 con 0x55c652757af0 2024-09-10T10:14:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b93fff640 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b937fe640 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275fb80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009d60 tx=0x7f9b88010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b0a4ddff6ef1b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b737fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b737fe640 1 -- 172.21.15.45:0/3906348742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8800fb90 con 0x55c652757af0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b737fe640 1 -- 172.21.15.45:0/3906348742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f9b8800fd30 con 0x55c652757af0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b84031870 con 0x55c652757af0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b737fe640 1 -- 172.21.15.45:0/3906348742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9b88005020 con 0x55c652757af0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b9d1c4740 10 client.15861 fetch_fsmap learned FSMap version 21 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c65249b620 con 0x55c652757af0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.535+0000 7f9b737fe640 1 -- 172.21.15.45:0/3906348742 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b84031870 con 0x55c652757af0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b737fe640 1 -- 172.21.15.45:0/3906348742 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9b88026480 con 0x55c652757af0 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b9d1c4740 10 client.15861 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b9d1c4740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:14:17.543 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b9d1c4740 20 client.15861 populate_metadata read hostname 'smithi045' 2024-09-10T10:14:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c652772c80 0x55c652775090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b9d1c4740 4 client.15861 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:14:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6526c9a20 con 0x55c652772c80 2024-09-10T10:14:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b93fff640 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c652772c80 0x55c652775090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b737fe640 10 client.15861 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:17.544 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.536+0000 7f9b93fff640 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c652772c80 0x55c652775090 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2b622ae3c95ee server_cookie=1cfa0b46540cb541 in_seq=0 out_seq=0 2024-09-10T10:14:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b737fe640 1 -- 172.21.15.45:0/3906348742 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6526c9a20 con 0x55c652772c80 2024-09-10T10:14:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b737fe640 10 client.15861 handle_command_reply: tid=0 2024-09-10T10:14:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 client.15861 shutdown 2024-09-10T10:14:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 msgr2=0x55c65275fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275fb80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009d60 tx=0x7f9b88010f30 comp rx=0 tx=0).stop 2024-09-10T10:14:17.545 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 shutdown_connections 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c652772c80 0x55c652775090 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c652760600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b737fe640 0 client.15861 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3906348742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275fb80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 >> 172.21.15.45:0/3906348742 conn(0x55c652754060 msgr2=0x55c652759dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.537+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 shutdown_connections 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.538+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3906348742 wait complete. 2024-09-10T10:14:17.546 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.538+0000 7f9b9d1c4740 20 client.15861 trim_cache size 0 max 16384 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 Processor -- start 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 -- start start 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c6527605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ab8b0 con 0x55c652757af0 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ae110 con 0x55c652758410 2024-09-10T10:14:17.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.552+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941abf00 con 0x55c652762cb0 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37384/0 (socket says 172.21.15.45:37384) 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 -- 172.21.15.45:0/1107748251 learned_addr learned my addr 172.21.15.45:0/1107748251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 -- 172.21.15.45:0/1107748251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 msgr2=0x55c6527605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c6527605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 -- 172.21.15.45:0/1107748251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 msgr2=0x55c6527600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b93fff640 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 -- 172.21.15.45:0/1107748251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c652757920 con 0x55c652757af0 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b93fff640 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b937fe640 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275a5b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b880268d0 tx=0x7f9b8800f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93cd5e34d0c5b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b79ffb640 1 -- 172.21.15.45:0/1107748251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88026480 con 0x55c652757af0 2024-09-10T10:14:17.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b79ffb640 1 -- 172.21.15.45:0/1107748251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b88026620 con 0x55c652757af0 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.553+0000 7f9b79ffb640 1 -- 172.21.15.45:0/1107748251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88005720 con 0x55c652757af0 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1107748251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 msgr2=0x55c65275a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275a5b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9b880268d0 tx=0x7f9b8800f820 comp rx=0 tx=0).stop 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1107748251 shutdown_connections 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652762cb0 0x55c6527605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652758410 0x55c6527600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/1107748251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65275a5b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1107748251 >> 172.21.15.45:0/1107748251 conn(0x55c652754060 msgr2=0x55c652761c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1107748251 shutdown_connections 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/1107748251 wait complete. 2024-09-10T10:14:17.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 -- start start 2024-09-10T10:14:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.554+0000 7f9b9d1c4740 1 Processor -- start 2024-09-10T10:14:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65276b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652758410 0x55c652766780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652762cb0 0x55c652766cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ac5d0 con 0x55c652757af0 2024-09-10T10:14:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941ada40 con 0x55c652762cb0 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941acca0 con 0x55c652758410 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 10 client.0 _ll_register_callbacks cb 0x7f9b88010810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65276b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65276b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37400/0 (socket says 172.21.15.45:37400) 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 -- 172.21.15.45:0/3284286399 learned_addr learned my addr 172.21.15.45:0/3284286399 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b93fff640 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652758410 0x55c652766780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 -- 172.21.15.45:0/3284286399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652758410 msgr2=0x55c652766780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652758410 0x55c652766780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 -- 172.21.15.45:0/3284286399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652762cb0 msgr2=0x55c652766cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652762cb0 0x55c652766cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 -- 172.21.15.45:0/3284286399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6527674f0 con 0x55c652757af0 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b937fe640 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65276b460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9b880098a0 tx=0x7f9b88010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52ec2d48d3a7a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b737fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b737fe640 1 -- 172.21.15.45:0/3284286399 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b88026480 con 0x55c652757af0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b737fe640 1 -- 172.21.15.45:0/3284286399 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f9b88026ac0 con 0x55c652757af0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b88026620 con 0x55c652757af0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b737fe640 1 -- 172.21.15.45:0/3284286399 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9b88026cc0 con 0x55c652757af0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.555+0000 7f9b9d1c4740 10 client.15867 fetch_fsmap learned FSMap version 21 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55c6527e1d60 con 0x55c652757af0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b737fe640 1 -- 172.21.15.45:0/3284286399 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b88026620 con 0x55c652757af0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b737fe640 1 -- 172.21.15.45:0/3284286399 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f9b8802f8c0 con 0x55c652757af0 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b9d1c4740 10 client.15867 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b9d1c4740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b9d1c4740 20 client.15867 populate_metadata read hostname 'smithi045' 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6527e2370 0x55c6527e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b9d1c4740 4 client.15867 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:14:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 --> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6526c3d70 con 0x55c6527e2370 2024-09-10T10:14:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.556+0000 7f9b93fff640 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6527e2370 0x55c6527e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.557+0000 7f9b93fff640 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6527e2370 0x55c6527e4760 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5475d934b96e91 server_cookie=552d603ed75f6d2e in_seq=0 out_seq=0 2024-09-10T10:14:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.557+0000 7f9b737fe640 10 client.15867 ms_handle_connect on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.564+0000 7f9b737fe640 1 -- 172.21.15.45:0/3284286399 <== mds.0 v2:172.21.15.47:6838/2511396298 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6526c3d70 con 0x55c6527e2370 2024-09-10T10:14:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.564+0000 7f9b737fe640 10 client.15867 handle_command_reply: tid=0 2024-09-10T10:14:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.564+0000 7f9b9d1c4740 1 client.15867 shutdown 2024-09-10T10:14:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 msgr2=0x55c65276b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65276b460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9b880098a0 tx=0x7f9b88010af0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 shutdown_connections 2024-09-10T10:14:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.47:6838/2511396298,v1:172.21.15.47:6839/2511396298] conn(0x55c6527e2370 0x55c6527e4760 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c652762cb0 0x55c652766cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x55c652758410 0x55c652766780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b737fe640 0 client.15867 ms_handle_reset on v2:172.21.15.47:6838/2511396298 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 --2- 172.21.15.45:0/3284286399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c652757af0 0x55c65276b460 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 >> 172.21.15.45:0/3284286399 conn(0x55c652754060 msgr2=0x55c6527529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 shutdown_connections 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 1 -- 172.21.15.45:0/3284286399 wait complete. 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9d1c4740 20 client.15867 trim_cache size 0 max 16384 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-09-10T10:14:17.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-09-10T10:14:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-09-10T10:14:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-09-10T10:14:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f9b7c0a2da0 msgr2=0x7f9b7c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f9b7c0a2da0 0x7f9b7c0a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c09bb60 tx=0x7f9b8c01bb30 comp rx=0 tx=0).stop 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 msgr2=0x7f9b9419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b9419ccc0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00eb10 tx=0x7f9b8c00c2d0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 shutdown_connections 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b941069c0 0x7f9b9419ccc0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f9b7c0a2da0 0x7f9b7c0a5260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b94101350 0x7f9b94198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 --2- 172.21.15.45:0/2918176630 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9b941009b0 0x7f9b94198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.565+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 >> 172.21.15.45:0/2918176630 conn(0x7f9b94077990 msgr2=0x7f9b94103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.566+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 shutdown_connections 2024-09-10T10:14:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-09-10T10:14:17.566+0000 7f9b9a34b640 1 -- 172.21.15.45:0/2918176630 wait complete. 2024-09-10T10:14:17.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:17 smithi162 ceph-mon[34963]: pgmap v345: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 663 B/s wr, 2 op/s 2024-09-10T10:14:17.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:17 smithi162 ceph-mon[34963]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T10:14:17.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:17 smithi045 ceph-mon[31114]: pgmap v345: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 663 B/s wr, 2 op/s 2024-09-10T10:14:17.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:17 smithi045 ceph-mon[31114]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T10:14:17.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:17 smithi047 ceph-mon[35244]: pgmap v345: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 663 B/s wr, 2 op/s 2024-09-10T10:14:17.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:17 smithi047 ceph-mon[35244]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T10:14:18.736 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T10:14:18.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T10:14:19.444 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:19 smithi045 ceph-mon[31114]: pgmap v347: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:14:19.445 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-10T10:14:19.446 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T10:14:19.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:19 smithi162 ceph-mon[34963]: pgmap v347: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:14:19.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:19 smithi047 ceph-mon[35244]: pgmap v347: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 409 B/s wr, 2 op/s 2024-09-10T10:14:19.734 DEBUG:teuthology.parallel:result is None 2024-09-10T10:14:19.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:14:19.767 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:14:19.767 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-10T10:14:19.777 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:14:19.815 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-10T10:14:19.832 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-10T10:14:19.832 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T10:14:19.832 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:14:19.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-10T10:14:19.883 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-10T10:14:19.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.093+0000 7fd6b7e42640 1 -- 172.21.15.45:0/3097246880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b0074620 msgr2=0x7fd6b0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.093+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/3097246880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b0074620 0x7fd6b0074a00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac009900 tx=0x7fd6ac02a480 comp rx=0 tx=0).stop 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 -- 172.21.15.45:0/3097246880 shutdown_connections 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/3097246880 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b010e320 0x7fd6b0110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/3097246880 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6b0074f40 0x7fd6b010dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/3097246880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b0074620 0x7fd6b0074a00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 -- 172.21.15.45:0/3097246880 >> 172.21.15.45:0/3097246880 conn(0x7fd6b00fd7e0 msgr2=0x7fd6b00ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 -- 172.21.15.45:0/3097246880 shutdown_connections 2024-09-10T10:14:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 -- 172.21.15.45:0/3097246880 wait complete. 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 Processor -- start 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.094+0000 7fd6b7e42640 1 -- start start 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b7e42640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6b0074620 0x7fd6b019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b7e42640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b0074f40 0x7fd6b019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b7e42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b010e320 0x7fd6b019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b7e42640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b01140f0 con 0x7fd6b010e320 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b7e42640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b0113f70 con 0x7fd6b0074620 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b7e42640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b0114270 con 0x7fd6b0074f40 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b010e320 0x7fd6b019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b010e320 0x7fd6b019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37424/0 (socket says 172.21.15.45:37424) 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 -- 172.21.15.45:0/1412056677 learned_addr learned my addr 172.21.15.45:0/1412056677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b53dd640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b0074f40 0x7fd6b019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b5bde640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6b0074620 0x7fd6b019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 -- 172.21.15.45:0/1412056677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b0074f40 msgr2=0x7fd6b019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b0074f40 0x7fd6b019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 -- 172.21.15.45:0/1412056677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6b0074620 msgr2=0x7fd6b019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6b0074620 0x7fd6b019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 -- 172.21.15.45:0/1412056677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b019f4d0 con 0x7fd6b010e320 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b63df640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b010e320 0x7fd6b019eda0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800eaa0 tx=0x7fd6a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6a6ffd640 1 -- 172.21.15.45:0/1412056677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a800cdb0 con 0x7fd6b010e320 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6a6ffd640 1 -- 172.21.15.45:0/1412056677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6a8005020 con 0x7fd6b010e320 2024-09-10T10:14:20.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b019f7c0 con 0x7fd6b010e320 2024-09-10T10:14:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.095+0000 7fd6a6ffd640 1 -- 172.21.15.45:0/1412056677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80116b0 con 0x7fd6b010e320 2024-09-10T10:14:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.096+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b01a7050 con 0x7fd6b010e320 2024-09-10T10:14:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.096+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b0074a00 con 0x7fd6b010e320 2024-09-10T10:14:20.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.098+0000 7fd6a6ffd640 1 -- 172.21.15.45:0/1412056677 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd6a8005560 con 0x7fd6b010e320 2024-09-10T10:14:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.098+0000 7fd6a6ffd640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd69c0a2e20 0x7fd69c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.098+0000 7fd6b5bde640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd69c0a2e20 0x7fd69c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.098+0000 7fd6a6ffd640 1 -- 172.21.15.45:0/1412056677 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fd6a80cf6b0 con 0x7fd6b010e320 2024-09-10T10:14:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.099+0000 7fd6b5bde640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd69c0a2e20 0x7fd69c0a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c00 tx=0x7fd6ac005a20 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:20.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.103+0000 7fd6a6ffd640 1 -- 172.21.15.45:0/1412056677 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6a8016030 con 0x7fd6b010e320 2024-09-10T10:14:20.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.300+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 --> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd6b00ffb00 con 0x7fd69c0a2e20 2024-09-10T10:14:20.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.310+0000 7fd6a6ffd640 1 -- 172.21.15.45:0/1412056677 <== mgr.15060 v2:172.21.15.45:6800/506901738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fd6b00ffb00 con 0x7fd69c0a2e20 2024-09-10T10:14:20.312 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T10:03:13", "first": "2024-09-10T10:04:00", "last": "2024-09-10T10:14:00", "last_pruned": "2024-09-10T10:14:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-09-10T10:14:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd69c0a2e20 msgr2=0x7fd69c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd69c0a2e20 0x7fd69c0a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c00 tx=0x7fd6ac005a20 comp rx=0 tx=0).stop 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b010e320 msgr2=0x7fd6b019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b010e320 0x7fd6b019eda0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800eaa0 tx=0x7fd6a800ef60 comp rx=0 tx=0).stop 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 shutdown_connections 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6b010e320 0x7fd6b019eda0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd6b0074f40 0x7fd6b019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd69c0a2e20 0x7fd69c0a52e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 --2- 172.21.15.45:0/1412056677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd6b0074620 0x7fd6b019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 >> 172.21.15.45:0/1412056677 conn(0x7fd6b00fd7e0 msgr2=0x7fd6b00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 shutdown_connections 2024-09-10T10:14:20.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.312+0000 7fd6b7e42640 1 -- 172.21.15.45:0/1412056677 wait complete. 2024-09-10T10:14:20.325 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T10:14:20.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 -- 172.21.15.45:0/1688432519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb484101370 msgr2=0x7fb48410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 --2- 172.21.15.45:0/1688432519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb484101370 0x7fb48410d8a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009cd0 tx=0x7fb47c02a8e0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 -- 172.21.15.45:0/1688432519 shutdown_connections 2024-09-10T10:14:20.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 --2- 172.21.15.45:0/1688432519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb484101370 0x7fb48410d8a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 --2- 172.21.15.45:0/1688432519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4841009d0 0x7fb484100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 --2- 172.21.15.45:0/1688432519 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb4841069e0 0x7fb484106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 -- 172.21.15.45:0/1688432519 >> 172.21.15.45:0/1688432519 conn(0x7fb4840fc6f0 msgr2=0x7fb4840feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:20.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 -- 172.21.15.45:0/1688432519 shutdown_connections 2024-09-10T10:14:20.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 -- 172.21.15.45:0/1688432519 wait complete. 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.428+0000 7fb489c3c640 1 Processor -- start 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb489c3c640 1 -- start start 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb489c3c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4841009d0 0x7fb484198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb489c3c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb484101370 0x7fb484198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb489c3c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4841069e0 0x7fb48419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb489c3c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb484110e20 con 0x7fb4841009d0 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb489c3c640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb484110ca0 con 0x7fb4841069e0 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb489c3c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb484110fa0 con 0x7fb484101370 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4841009d0 0x7fb484198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4841009d0 0x7fb484198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37448/0 (socket says 172.21.15.45:37448) 2024-09-10T10:14:20.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 -- 172.21.15.45:0/2516276778 learned_addr learned my addr 172.21.15.45:0/2516276778 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb483fff640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4841069e0 0x7fb48419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 -- 172.21.15.45:0/2516276778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb484101370 msgr2=0x7fb484198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb484101370 0x7fb484198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 -- 172.21.15.45:0/2516276778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4841069e0 msgr2=0x7fb48419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4841069e0 0x7fb48419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 -- 172.21.15.45:0/2516276778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48419d310 con 0x7fb4841009d0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb483fff640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4841069e0 0x7fb48419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb4837fe640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4841009d0 0x7fb484198320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb47400d770 tx=0x7fb47400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.429+0000 7fb480ff9640 1 -- 172.21.15.45:0/2516276778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474017070 con 0x7fb4841009d0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.430+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48419d600 con 0x7fb4841009d0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.430+0000 7fb480ff9640 1 -- 172.21.15.45:0/2516276778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb474005500 con 0x7fb4841009d0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.430+0000 7fb480ff9640 1 -- 172.21.15.45:0/2516276778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474004e00 con 0x7fb4841009d0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.430+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4841a4e90 con 0x7fb4841009d0 2024-09-10T10:14:20.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.431+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb484102110 con 0x7fb4841009d0 2024-09-10T10:14:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.432+0000 7fb480ff9640 1 -- 172.21.15.45:0/2516276778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb474004fa0 con 0x7fb4841009d0 2024-09-10T10:14:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.432+0000 7fb480ff9640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb46c0a2da0 0x7fb46c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.432+0000 7fb480ff9640 1 -- 172.21.15.45:0/2516276778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fb4740ceee0 con 0x7fb4841009d0 2024-09-10T10:14:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.433+0000 7fb482ffd640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb46c0a2da0 0x7fb46c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.433+0000 7fb482ffd640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb46c0a2da0 0x7fb46c0a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb478005fd0 tx=0x7fb478005850 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:20.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.437+0000 7fb480ff9640 1 -- 172.21.15.45:0/2516276778 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb47409bae0 con 0x7fb4841009d0 2024-09-10T10:14:20.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.635+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 --> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb484109fc0 con 0x7fb46c0a2da0 2024-09-10T10:14:20.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.642+0000 7fb480ff9640 1 -- 172.21.15.45:0/2516276778 <== mgr.15060 v2:172.21.15.45:6800/506901738 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb47800d9f0 con 0x7fb46c0a2da0 2024-09-10T10:14:20.645 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb46c0a2da0 msgr2=0x7fb46c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb46c0a2da0 0x7fb46c0a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb478005fd0 tx=0x7fb478005850 comp rx=0 tx=0).stop 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4841009d0 msgr2=0x7fb484198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4841009d0 0x7fb484198320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb47400d770 tx=0x7fb47400dc30 comp rx=0 tx=0).stop 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 shutdown_connections 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb46c0a2da0 0x7fb46c0a5260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb4841069e0 0x7fb48419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb484101370 0x7fb484198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 --2- 172.21.15.45:0/2516276778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb4841009d0 0x7fb484198320 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 >> 172.21.15.45:0/2516276778 conn(0x7fb4840fc6f0 msgr2=0x7fb48406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 shutdown_connections 2024-09-10T10:14:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.644+0000 7fb489c3c640 1 -- 172.21.15.45:0/2516276778 wait complete. 2024-09-10T10:14:20.658 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T10:14:20.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:14:20.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 -- 172.21.15.45:0/1266658412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08103670 msgr2=0x7fda0810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 --2- 172.21.15.45:0/1266658412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08103670 0x7fda0810bb50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009900 tx=0x7fd9f802a480 comp rx=0 tx=0).stop 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 -- 172.21.15.45:0/1266658412 shutdown_connections 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 --2- 172.21.15.45:0/1266658412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda0810c120 0x7fda0810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 --2- 172.21.15.45:0/1266658412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08103670 0x7fda0810bb50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 --2- 172.21.15.45:0/1266658412 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda08102d50 0x7fda08103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 -- 172.21.15.45:0/1266658412 >> 172.21.15.45:0/1266658412 conn(0x7fda080fc6a0 msgr2=0x7fda080feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 -- 172.21.15.45:0/1266658412 shutdown_connections 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.768+0000 7fda103af640 1 -- 172.21.15.45:0/1266658412 wait complete. 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 Processor -- start 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 -- start start 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08102d50 0x7fda081009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda08103670 0x7fda08100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda0810c120 0x7fda08101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fda08111ec0 con 0x7fda08102d50 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fda08111d40 con 0x7fda0810c120 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda103af640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fda08112040 con 0x7fda08103670 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda0e14b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08102d50 0x7fda081009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda0e14b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08102d50 0x7fda081009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37464/0 (socket says 172.21.15.45:37464) 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda0e14b640 1 -- 172.21.15.45:0/2667795460 learned_addr learned my addr 172.21.15.45:0/2667795460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda0e94c640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda0810c120 0x7fda08101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.769+0000 7fda0d94a640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda08103670 0x7fda08100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda0e14b640 1 -- 172.21.15.45:0/2667795460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda08103670 msgr2=0x7fda08100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda0e14b640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda08103670 0x7fda08100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda0e14b640 1 -- 172.21.15.45:0/2667795460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda0810c120 msgr2=0x7fda08101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda0e14b640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda0810c120 0x7fda08101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda0e14b640 1 -- 172.21.15.45:0/2667795460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda081a6950 con 0x7fda08102d50 2024-09-10T10:14:20.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda0e14b640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08102d50 0x7fda081009a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fda0400c970 tx=0x7fda0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:20.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fd9ff7fe640 1 -- 172.21.15.45:0/2667795460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04007bb0 con 0x7fda08102d50 2024-09-10T10:14:20.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fd9ff7fe640 1 -- 172.21.15.45:0/2667795460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda04007d50 con 0x7fda08102d50 2024-09-10T10:14:20.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fd9ff7fe640 1 -- 172.21.15.45:0/2667795460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0400f450 con 0x7fda08102d50 2024-09-10T10:14:20.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda081a6c40 con 0x7fda08102d50 2024-09-10T10:14:20.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.770+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda081a7190 con 0x7fda08102d50 2024-09-10T10:14:20.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.771+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda08103130 con 0x7fda08102d50 2024-09-10T10:14:20.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.772+0000 7fd9ff7fe640 1 -- 172.21.15.45:0/2667795460 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fda04005020 con 0x7fda08102d50 2024-09-10T10:14:20.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.773+0000 7fd9ff7fe640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd9f40a2d80 0x7fd9f40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:20.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.773+0000 7fd9ff7fe640 1 -- 172.21.15.45:0/2667795460 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fda040cf240 con 0x7fda08102d50 2024-09-10T10:14:20.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.773+0000 7fda0d94a640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd9f40a2d80 0x7fd9f40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:20.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.774+0000 7fda0d94a640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd9f40a2d80 0x7fd9f40a5240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda08102430 tx=0x7fd9f8005a20 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:20.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.777+0000 7fd9ff7fe640 1 -- 172.21.15.45:0/2667795460 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda04014030 con 0x7fda08102d50 2024-09-10T10:14:20.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.955+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda0802d7c0 con 0x7fda08102d50 2024-09-10T10:14:20.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.957+0000 7fd9ff7fe640 1 -- 172.21.15.45:0/2667795460 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23198 (secure 0 0 0) 0x7fda0409bde0 con 0x7fda08102d50 2024-09-10T10:14:20.959 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:14:20.960 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:14:16.271892+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/952575454":"2024-09-11T10:03:11.647760+0000","172.21.15.45:6800/952575454":"2024-09-11T10:03:11.647760+0000","172.21.15.45:0/3649460339":"2024-09-11T10:03:11.647760+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/384122965":"2024-09-11T10:03:11.647760+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/689699829":"2024-09-11T10:03:11.647760+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/375260435":"2024-09-11T10:03:11.647760+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/21896051":"2024-09-11T10:03:11.647760+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:14:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.958+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd9f40a2d80 msgr2=0x7fd9f40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.958+0000 7fda103af640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd9f40a2d80 0x7fd9f40a5240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda08102430 tx=0x7fd9f8005a20 comp rx=0 tx=0).stop 2024-09-10T10:14:20.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08102d50 msgr2=0x7fda081009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08102d50 0x7fda081009a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fda0400c970 tx=0x7fda0400ce30 comp rx=0 tx=0).stop 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 shutdown_connections 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fd9f40a2d80 0x7fd9f40a5240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda0810c120 0x7fda08101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fda08103670 0x7fda08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 --2- 172.21.15.45:0/2667795460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fda08102d50 0x7fda081009a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 >> 172.21.15.45:0/2667795460 conn(0x7fda080fc6a0 msgr2=0x7fda080fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 shutdown_connections 2024-09-10T10:14:20.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:20.959+0000 7fda103af640 1 -- 172.21.15.45:0/2667795460 wait complete. 2024-09-10T10:14:20.973 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:57:03.045230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:01:30.706328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:01:33.219365+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:14:20.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:14:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 -- 172.21.15.45:0/2849600839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f810c070 msgr2=0x7f56f810e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 --2- 172.21.15.45:0/2849600839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f810c070 0x7f56f810e530 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009ca0 tx=0x7f56f002a950 comp rx=0 tx=0).stop 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 -- 172.21.15.45:0/2849600839 shutdown_connections 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 --2- 172.21.15.45:0/2849600839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f810c070 0x7f56f810e530 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 --2- 172.21.15.45:0/2849600839 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56f8103600 0x7f56f810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 --2- 172.21.15.45:0/2849600839 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f8102ce0 0x7f56f81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 -- 172.21.15.45:0/2849600839 >> 172.21.15.45:0/2849600839 conn(0x7f56f80fc6b0 msgr2=0x7f56f80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 -- 172.21.15.45:0/2849600839 shutdown_connections 2024-09-10T10:14:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.077+0000 7f56fe477640 1 -- 172.21.15.45:0/2849600839 wait complete. 2024-09-10T10:14:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 Processor -- start 2024-09-10T10:14:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 -- start start 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56f8102ce0 0x7f56f81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8103600 0x7f56f8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f810c070 0x7f56f819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8111e90 con 0x7f56f8103600 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8111d10 con 0x7f56f8102ce0 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56fe477640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f56f8112010 con 0x7f56f810c070 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8103600 0x7f56f8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56f77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8103600 0x7f56f8198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37476/0 (socket says 172.21.15.45:37476) 2024-09-10T10:14:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56f77fe640 1 -- 172.21.15.45:0/845587209 learned_addr learned my addr 172.21.15.45:0/845587209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.078+0000 7f56f7fff640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56f8102ce0 0x7f56f81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56fca14640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f810c070 0x7f56f819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f77fe640 1 -- 172.21.15.45:0/845587209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f810c070 msgr2=0x7f56f819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f77fe640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f810c070 0x7f56f819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f77fe640 1 -- 172.21.15.45:0/845587209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56f8102ce0 msgr2=0x7f56f81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f77fe640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56f8102ce0 0x7f56f81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f77fe640 1 -- 172.21.15.45:0/845587209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f819d2d0 con 0x7f56f8103600 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f7fff640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56f8102ce0 0x7f56f81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f77fe640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8103600 0x7f56f8198820 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c920 tx=0x7f56ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f57fa640 1 -- 172.21.15.45:0/845587209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec007c70 con 0x7f56f8103600 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f57fa640 1 -- 172.21.15.45:0/845587209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ec007e10 con 0x7f56f8103600 2024-09-10T10:14:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56f57fa640 1 -- 172.21.15.45:0/845587209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec018890 con 0x7f56f8103600 2024-09-10T10:14:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f819d5c0 con 0x7f56f8103600 2024-09-10T10:14:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.079+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f81a4e50 con 0x7f56f8103600 2024-09-10T10:14:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.080+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f81030c0 con 0x7f56f8103600 2024-09-10T10:14:21.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.081+0000 7f56f57fa640 1 -- 172.21.15.45:0/845587209 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f56ec018a70 con 0x7f56f8103600 2024-09-10T10:14:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.082+0000 7f56f57fa640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f56e00a2d00 0x7f56e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.082+0000 7f56f7fff640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f56e00a2d00 0x7f56e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.082+0000 7f56f57fa640 1 -- 172.21.15.45:0/845587209 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f56ec0cf5a0 con 0x7f56f8103600 2024-09-10T10:14:21.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.083+0000 7f56f7fff640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f56e00a2d00 0x7f56e00a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005fd0 tx=0x7f56e800b390 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:21.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.086+0000 7f56f57fa640 1 -- 172.21.15.45:0/845587209 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56ec014030 con 0x7f56f8103600 2024-09-10T10:14:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.263+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56f81a5180 con 0x7f56f8103600 2024-09-10T10:14:21.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.264+0000 7f56f57fa640 1 -- 172.21.15.45:0/845587209 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f56ec09c140 con 0x7f56f8103600 2024-09-10T10:14:21.266 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-09-10T10:14:21.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f56e00a2d00 msgr2=0x7f56e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f56e00a2d00 0x7f56e00a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56e8005fd0 tx=0x7f56e800b390 comp rx=0 tx=0).stop 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8103600 msgr2=0x7f56f8198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8103600 0x7f56f8198820 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c920 tx=0x7f56ec00cde0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 shutdown_connections 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f56f810c070 0x7f56f819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56f8103600 0x7f56f8198820 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f56e00a2d00 0x7f56e00a51c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 --2- 172.21.15.45:0/845587209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56f8102ce0 0x7f56f81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 >> 172.21.15.45:0/845587209 conn(0x7f56f80fc6b0 msgr2=0x7f56f80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 shutdown_connections 2024-09-10T10:14:21.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.266+0000 7f56fe477640 1 -- 172.21.15.45:0/845587209 wait complete. 2024-09-10T10:14:21.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T10:14:21.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 -- 172.21.15.45:0/443176780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc10c0d0 msgr2=0x7f41dc10e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 --2- 172.21.15.45:0/443176780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc10c0d0 0x7f41dc10e550 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009ca0 tx=0x7f41d402a950 comp rx=0 tx=0).stop 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 -- 172.21.15.45:0/443176780 shutdown_connections 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 --2- 172.21.15.45:0/443176780 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc10c0d0 0x7f41dc10e550 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 --2- 172.21.15.45:0/443176780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41dc1035a0 0x7f41dc10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 --2- 172.21.15.45:0/443176780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41dc102c80 0x7f41dc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 -- 172.21.15.45:0/443176780 >> 172.21.15.45:0/443176780 conn(0x7f41dc0fc690 msgr2=0x7f41dc0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 -- 172.21.15.45:0/443176780 shutdown_connections 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 -- 172.21.15.45:0/443176780 wait complete. 2024-09-10T10:14:21.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.384+0000 7f41e16af640 1 Processor -- start 2024-09-10T10:14:21.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41e16af640 1 -- start start 2024-09-10T10:14:21.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41e16af640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41dc102c80 0x7f41dc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41e16af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc1035a0 0x7f41dc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41e16af640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41dc10c0d0 0x7f41dc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41e16af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc111e90 con 0x7f41dc1035a0 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41e16af640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc111d10 con 0x7f41dc102c80 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41e16af640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc112010 con 0x7f41dc10c0d0 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc1035a0 0x7f41dc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc1035a0 0x7f41dc1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37504/0 (socket says 172.21.15.45:37504) 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 -- 172.21.15.45:0/134825144 learned_addr learned my addr 172.21.15.45:0/134825144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41daffd640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41dc102c80 0x7f41dc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41db7fe640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41dc10c0d0 0x7f41dc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 -- 172.21.15.45:0/134825144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41dc10c0d0 msgr2=0x7f41dc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41dc10c0d0 0x7f41dc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 -- 172.21.15.45:0/134825144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41dc102c80 msgr2=0x7f41dc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41dc102c80 0x7f41dc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41da7fc640 1 -- 172.21.15.45:0/134825144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc19d460 con 0x7f41dc1035a0 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41db7fe640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41dc10c0d0 0x7f41dc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.385+0000 7f41daffd640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41dc102c80 0x7f41dc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.386+0000 7f41da7fc640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc1035a0 0x7f41dc1989b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f41d000d750 tx=0x7f41d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.386+0000 7f41c3fff640 1 -- 172.21.15.45:0/134825144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0017070 con 0x7f41dc1035a0 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.386+0000 7f41c3fff640 1 -- 172.21.15.45:0/134825144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41d0005020 con 0x7f41dc1035a0 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.386+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc19d750 con 0x7f41dc1035a0 2024-09-10T10:14:21.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.386+0000 7f41c3fff640 1 -- 172.21.15.45:0/134825144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0004d00 con 0x7f41dc1035a0 2024-09-10T10:14:21.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.386+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1a4fe0 con 0x7f41dc1035a0 2024-09-10T10:14:21.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.387+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc103060 con 0x7f41dc1035a0 2024-09-10T10:14:21.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.388+0000 7f41c3fff640 1 -- 172.21.15.45:0/134825144 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f41d0009a30 con 0x7f41dc1035a0 2024-09-10T10:14:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.389+0000 7f41c3fff640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41c40a2d00 0x7f41c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.389+0000 7f41daffd640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41c40a2d00 0x7f41c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.389+0000 7f41c3fff640 1 -- 172.21.15.45:0/134825144 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f41d00cee40 con 0x7f41dc1035a0 2024-09-10T10:14:21.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.389+0000 7f41daffd640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41c40a2d00 0x7f41c40a51c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00bfd0 tx=0x7f41cc00b390 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:21.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.393+0000 7f41c3fff640 1 -- 172.21.15.45:0/134825144 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41d009b9e0 con 0x7f41dc1035a0 2024-09-10T10:14:21.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.570+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41dc107120 con 0x7f41dc1035a0 2024-09-10T10:14:21.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.570+0000 7f41c3fff640 1 -- 172.21.15.45:0/134825144 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f41d009b2e0 con 0x7f41dc1035a0 2024-09-10T10:14:21.573 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41c40a2d00 msgr2=0x7f41c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41c40a2d00 0x7f41c40a51c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00bfd0 tx=0x7f41cc00b390 comp rx=0 tx=0).stop 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc1035a0 msgr2=0x7f41dc1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc1035a0 0x7f41dc1989b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f41d000d750 tx=0x7f41d000dc10 comp rx=0 tx=0).stop 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 shutdown_connections 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f41dc10c0d0 0x7f41dc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41dc1035a0 0x7f41dc1989b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f41c40a2d00 0x7f41c40a51c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 --2- 172.21.15.45:0/134825144 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41dc102c80 0x7f41dc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 >> 172.21.15.45:0/134825144 conn(0x7f41dc0fc690 msgr2=0x7f41dc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:21.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 shutdown_connections 2024-09-10T10:14:21.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.573+0000 7f41e16af640 1 -- 172.21.15.45:0/134825144 wait complete. 2024-09-10T10:14:21.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:21 smithi045 ceph-mon[31114]: pgmap v348: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 26 KiB/s wr, 3 op/s 2024-09-10T10:14:21.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:21 smithi045 ceph-mon[31114]: from='client.15873 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:14:21.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:21 smithi045 ceph-mon[31114]: from='client.15879 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:14:21.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:21 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2667795460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:14:21.585 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:21 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/845587209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:14:21.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T10:14:21.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:21 smithi162 ceph-mon[34963]: pgmap v348: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 26 KiB/s wr, 3 op/s 2024-09-10T10:14:21.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:21 smithi162 ceph-mon[34963]: from='client.15873 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:14:21.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:21 smithi162 ceph-mon[34963]: from='client.15879 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:14:21.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:21 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2667795460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:14:21.651 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:21 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/845587209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:14:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:21 smithi047 ceph-mon[35244]: pgmap v348: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 26 KiB/s wr, 3 op/s 2024-09-10T10:14:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:21 smithi047 ceph-mon[35244]: from='client.15873 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:14:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:21 smithi047 ceph-mon[35244]: from='client.15879 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T10:14:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:21 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2667795460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:14:21.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:21 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/845587209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:14:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 -- 172.21.15.45:0/2593249012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 msgr2=0x7fef8c10e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 --2- 172.21.15.45:0/2593249012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c10e530 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ca0 tx=0x7fef8402a950 comp rx=0 tx=0).stop 2024-09-10T10:14:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 -- 172.21.15.45:0/2593249012 shutdown_connections 2024-09-10T10:14:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 --2- 172.21.15.45:0/2593249012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c10e530 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 --2- 172.21.15.45:0/2593249012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef8c103610 0x7fef8c10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 --2- 172.21.15.45:0/2593249012 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef8c102cf0 0x7fef8c1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 -- 172.21.15.45:0/2593249012 >> 172.21.15.45:0/2593249012 conn(0x7fef8c0779d0 msgr2=0x7fef8c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 -- 172.21.15.45:0/2593249012 shutdown_connections 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 -- 172.21.15.45:0/2593249012 wait complete. 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.691+0000 7fef91eba640 1 Processor -- start 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef91eba640 1 -- start start 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef91eba640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef8c102cf0 0x7fef8c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef91eba640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef8c103610 0x7fef8c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef91eba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef91eba640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c111e90 con 0x7fef8c10c0c0 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef91eba640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c111d10 con 0x7fef8c103610 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef91eba640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fef8c112010 con 0x7fef8c102cf0 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37520/0 (socket says 172.21.15.45:37520) 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 -- 172.21.15.45:0/2050791765 learned_addr learned my addr 172.21.15.45:0/2050791765 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8affd640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef8c103610 0x7fef8c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8b7fe640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef8c102cf0 0x7fef8c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 -- 172.21.15.45:0/2050791765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef8c102cf0 msgr2=0x7fef8c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef8c102cf0 0x7fef8c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 -- 172.21.15.45:0/2050791765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef8c103610 msgr2=0x7fef8c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef8c103610 0x7fef8c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 -- 172.21.15.45:0/2050791765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8c19d350 con 0x7fef8c10c0c0 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.692+0000 7fef8bfff640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c19cc20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fef84004af0 tx=0x7fef8402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.693+0000 7fef88ff9640 1 -- 172.21.15.45:0/2050791765 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84034040 con 0x7fef8c10c0c0 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.693+0000 7fef88ff9640 1 -- 172.21.15.45:0/2050791765 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef84030920 con 0x7fef8c10c0c0 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.693+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8c19d5e0 con 0x7fef8c10c0c0 2024-09-10T10:14:21.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.693+0000 7fef88ff9640 1 -- 172.21.15.45:0/2050791765 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84004820 con 0x7fef8c10c0c0 2024-09-10T10:14:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.693+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8c1a4ed0 con 0x7fef8c10c0c0 2024-09-10T10:14:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.694+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8c1030d0 con 0x7fef8c10c0c0 2024-09-10T10:14:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.695+0000 7fef88ff9640 1 -- 172.21.15.45:0/2050791765 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef84037020 con 0x7fef8c10c0c0 2024-09-10T10:14:21.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.696+0000 7fef88ff9640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fef740a2d80 0x7fef740a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.696+0000 7fef8b7fe640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fef740a2d80 0x7fef740a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.696+0000 7fef88ff9640 1 -- 172.21.15.45:0/2050791765 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fef840f0250 con 0x7fef8c10c0c0 2024-09-10T10:14:21.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.696+0000 7fef8b7fe640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fef740a2d80 0x7fef740a5240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0058c0 tx=0x7fef7c00a4c0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:21.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.700+0000 7fef88ff9640 1 -- 172.21.15.45:0/2050791765 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef840bcf60 con 0x7fef8c10c0c0 2024-09-10T10:14:21.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.878+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef8c107170 con 0x7fef8c10c0c0 2024-09-10T10:14:21.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.878+0000 7fef88ff9640 1 -- 172.21.15.45:0/2050791765 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fef840bc860 con 0x7fef8c10c0c0 2024-09-10T10:14:21.881 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fef740a2d80 msgr2=0x7fef740a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fef740a2d80 0x7fef740a5240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef7c0058c0 tx=0x7fef7c00a4c0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 msgr2=0x7fef8c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c19cc20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fef84004af0 tx=0x7fef8402f5d0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 shutdown_connections 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef8c10c0c0 0x7fef8c19cc20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fef740a2d80 0x7fef740a5240 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef8c103610 0x7fef8c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 --2- 172.21.15.45:0/2050791765 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fef8c102cf0 0x7fef8c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.880+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 >> 172.21.15.45:0/2050791765 conn(0x7fef8c0779d0 msgr2=0x7fef8c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.881+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 shutdown_connections 2024-09-10T10:14:21.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.881+0000 7fef91eba640 1 -- 172.21.15.45:0/2050791765 wait complete. 2024-09-10T10:14:21.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T10:14:22.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.995+0000 7f5582f31640 1 -- 172.21.15.45:0/1536544730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c100880 msgr2=0x7f557c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:22.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.995+0000 7f5582f31640 1 --2- 172.21.15.45:0/1536544730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c100880 0x7f557c100c60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f556c009900 tx=0x7f556c02a410 comp rx=0 tx=0).stop 2024-09-10T10:14:22.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.995+0000 7f5582f31640 1 -- 172.21.15.45:0/1536544730 shutdown_connections 2024-09-10T10:14:22.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 --2- 172.21.15.45:0/1536544730 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557c1080d0 0x7f557c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:22.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 --2- 172.21.15.45:0/1536544730 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f557c101230 0x7f557c107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:22.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 --2- 172.21.15.45:0/1536544730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c100880 0x7f557c100c60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 -- 172.21.15.45:0/1536544730 >> 172.21.15.45:0/1536544730 conn(0x7f557c075f60 msgr2=0x7f557c076370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 -- 172.21.15.45:0/1536544730 shutdown_connections 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 -- 172.21.15.45:0/1536544730 wait complete. 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 Processor -- start 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 -- start start 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f557c100880 0x7f557c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557c1080d0 0x7f557c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c19ccf0 0x7f557c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f557c111ee0 con 0x7f557c19ccf0 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f557c111d60 con 0x7f557c1080d0 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.996+0000 7f5582f31640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f557c112060 con 0x7f557c100880 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c19ccf0 0x7f557c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:22.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c19ccf0 0x7f557c19d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37544/0 (socket says 172.21.15.45:37544) 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 -- 172.21.15.45:0/4248329187 learned_addr learned my addr 172.21.15.45:0/4248329187 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f557bfff640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557c1080d0 0x7f557c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f5580ccd640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f557c100880 0x7f557c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 -- 172.21.15.45:0/4248329187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f557c100880 msgr2=0x7f557c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f557c100880 0x7f557c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 -- 172.21.15.45:0/4248329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557c1080d0 msgr2=0x7f557c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557c1080d0 0x7f557c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 -- 172.21.15.45:0/4248329187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c19d7e0 con 0x7f557c19ccf0 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f55814ce640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c19ccf0 0x7f557c19d1a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5574009fd0 tx=0x7f557400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f5579ffb640 1 -- 172.21.15.45:0/4248329187 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574017070 con 0x7f557c19ccf0 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f5579ffb640 1 -- 172.21.15.45:0/4248329187 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5574003e50 con 0x7f557c19ccf0 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c19dad0 con 0x7f557c19ccf0 2024-09-10T10:14:22.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.997+0000 7f5579ffb640 1 -- 172.21.15.45:0/4248329187 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55740106c0 con 0x7f557c19ccf0 2024-09-10T10:14:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.998+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c1a5270 con 0x7f557c19ccf0 2024-09-10T10:14:22.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.998+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c100c60 con 0x7f557c19ccf0 2024-09-10T10:14:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:21.999+0000 7f5579ffb640 1 -- 172.21.15.45:0/4248329187 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5574003970 con 0x7f557c19ccf0 2024-09-10T10:14:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.000+0000 7f5579ffb640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f55640a2d00 0x7f55640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.000+0000 7f5579ffb640 1 -- 172.21.15.45:0/4248329187 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f55740cf200 con 0x7f557c19ccf0 2024-09-10T10:14:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.000+0000 7f5580ccd640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f55640a2d00 0x7f55640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:22.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.001+0000 7f5580ccd640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f55640a2d00 0x7f55640a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f556c005ab0 tx=0x7f556c005a20 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:22.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.004+0000 7f5579ffb640 1 -- 172.21.15.45:0/4248329187 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f557409bda0 con 0x7f557c19ccf0 2024-09-10T10:14:22.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.232+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f557c101230 con 0x7f557c19ccf0 2024-09-10T10:14:22.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.240+0000 7f5579ffb640 1 -- 172.21.15.45:0/4248329187 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740899 (secure 0 0 0) 0x7f557409b6a0 con 0x7f557c19ccf0 2024-09-10T10:14:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:14:22.252 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T10:14:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 25, 2024-09-10T10:14:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.252 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 15060, 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 506901738 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 506901738 2024-09-10T10:14:22.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/506901738", 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2024-09-10T10:03:11.647977+0000", 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24739, 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:14:22.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.256 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.258 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.259 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-09-10T10:14:22.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.269 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:14:22.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.278 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.280 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.281 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.284 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-09-10T10:14:22.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-09-10T10:14:22.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:14:22.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.289 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:14:22.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.297 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.298 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.299 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.300 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.306 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:14:22.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:14:22.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:14:22.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:14:22.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:14:22.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:14:22.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:14:22.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.329 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.330 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.331 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.332 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-09-10T10:14:22.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.339 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.340 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:14:22.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.355 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.357 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.359 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.361 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.362 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.368 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.387 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.404 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-09-10T10:14:22.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-09-10T10:14:22.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-09-10T10:14:22.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.410 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.418 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-09-10T10:14:22.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-10T10:14:22.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.428 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:14:22.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.440 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-09-10T10:14:22.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-09-10T10:14:22.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:14:22.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.463 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.466 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.476 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.478 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.480 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-09-10T10:14:22.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-09-10T10:14:22.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.489 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.490 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.494 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.495 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.503 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.507 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-10T10:14:22.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.513 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.514 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.518 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.519 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.523 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.524 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:14:22.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:14:22.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:14:22.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.545 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-10T10:14:22.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-09-10T10:14:22.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-10T10:14:22.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.557 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.558 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.559 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-10T10:14:22.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.567 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.577 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.580 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.581 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.582 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-09-10T10:14:22.585 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.586 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.588 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.589 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-09-10T10:14:22.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.594 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.602 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.605 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.607 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.610 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.611 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.614 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.615 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.616 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.618 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.620 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:14:22.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.623 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.624 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.626 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.628 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.632 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.633 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.635 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.641 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.648 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.650 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.652 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.654 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.655 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-09-10T10:14:22.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-09-10T10:14:22.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:14:22.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.665 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.667 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.668 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.673 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.674 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 24818, 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:22.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.679 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-09-10T10:14:22.681 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-09-10T10:14:22.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-09-10T10:14:22.685 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.688 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.689 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.690 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.691 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-09-10T10:14:22.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-09-10T10:14:22.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-09-10T10:14:22.694 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-09-10T10:14:22.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-09-10T10:14:22.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.698 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.700 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.701 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.703 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-09-10T10:14:22.704 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:22.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:14:22.710 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.712 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:14:22.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.714 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-09-10T10:14:22.716 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.717 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.718 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:14:22.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.722 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.728 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-09-10T10:14:22.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.730 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:14:22.732 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.734 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-09-10T10:14:22.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.745 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.749 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:14:22.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:14:22.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.754 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-10T10:14:22.755 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.756 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:14:22.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:14:22.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.772 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.774 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:14:22.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:14:22.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.779 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.781 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.784 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:14:22.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:14:22.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:22.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.794 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:14:22.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:14:22.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:14:22.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:14:22.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.802 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.807 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:22.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.816 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.823 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.827 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.828 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.829 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:22.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-10T10:14:22.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-09-10T10:14:22.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.834 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-09-10T10:14:22.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.838 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.843 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.844 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.845 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-09-10T10:14:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-09-10T10:14:22.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.849 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.850 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.852 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.857 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.862 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.873 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.876 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.878 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.881 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.890 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.893 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:14:22.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:14:22.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.902 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.903 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.904 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.909 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-09-10T10:14:22.911 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.912 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.914 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.915 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-10T10:14:22.916 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.918 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.920 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.923 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.924 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.928 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.929 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.931 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.933 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.934 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.936 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.948 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.949 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.951 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.955 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:14:22.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-09-10T10:14:22.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.958 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-09-10T10:14:22.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.963 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.964 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.965 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.966 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.973 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.974 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.976 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.977 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:22.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.980 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.982 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-09-10T10:14:22.984 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.985 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.987 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.988 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:22.994 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:22.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-09-10T10:14:23.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.003 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-09-10T10:14:23.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.010 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.012 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.013 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.014 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.015 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.017 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.022 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.024 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.029 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.032 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:14:23.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.037 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.041 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.045 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.047 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-09-10T10:14:23.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-09-10T10:14:23.050 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.051 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.053 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.055 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.056 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.062 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-09-10T10:14:23.064 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.069 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.074 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "restful", 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_schedule" 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-09-10T10:14:23.075 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.077 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.081 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-09-10T10:14:23.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-09-10T10:14:23.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.089 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-09-10T10:14:23.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-09-10T10:14:23.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.092 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.094 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-09-10T10:14:23.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-09-10T10:14:23.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.097 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.098 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-09-10T10:14:23.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-10T10:14:23.103 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-09-10T10:14:23.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:14:23.105 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.108 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-09-10T10:14:23.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:23.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-09-10T10:14:23.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.118 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.124 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:14:23.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-09-10T10:14:23.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-09-10T10:14:23.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-09-10T10:14:23.129 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-09-10T10:14:23.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.132 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:14:23.134 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:23.136 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-09-10T10:14:23.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.142 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:14:23.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.146 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:14:23.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.149 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:14:23.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.156 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:14:23.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.159 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.160 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-09-10T10:14:23.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-09-10T10:14:23.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-09-10T10:14:23.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.167 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.172 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.174 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.175 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.176 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.178 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.179 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.180 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.181 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.186 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.187 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.191 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.203 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.204 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.205 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:14:23.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.216 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:14:23.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:14:23.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.228 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-09-10T10:14:23.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:14:23.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-09-10T10:14:23.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-10T10:14:23.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-09-10T10:14:23.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.243 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.244 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.245 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.249 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.250 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:23.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-09-10T10:14:23.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.254 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-09-10T10:14:23.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.271 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.272 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-09-10T10:14:23.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-09-10T10:14:23.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.277 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.280 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.281 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.282 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.286 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:14:23.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:14:23.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.289 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.293 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-09-10T10:14:23.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.297 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.298 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-09-10T10:14:23.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.305 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.306 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-09-10T10:14:23.310 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.311 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.316 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.319 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.322 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-09-10T10:14:23.327 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.328 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.329 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.334 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.337 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.339 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.342 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:23.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-09-10T10:14:23.345 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.347 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.350 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-09-10T10:14:23.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.359 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-09-10T10:14:23.362 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.369 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.373 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.379 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.380 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.383 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-09-10T10:14:23.389 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.390 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.398 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-09-10T10:14:23.401 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.402 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-09-10T10:14:23.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-09-10T10:14:23.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.420 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.425 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.430 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:14:23.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.437 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:14:23.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.446 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-09-10T10:14:23.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-09-10T10:14:23.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.454 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-09-10T10:14:23.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.461 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.462 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.464 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.466 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.469 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-09-10T10:14:23.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-09-10T10:14:23.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.480 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.484 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.485 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-10T10:14:23.486 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-10T10:14:23.487 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-10T10:14:23.488 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "squid": [ 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-09-10T10:14:23.489 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 84, 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2071516080 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3391462677 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1205016502 2024-09-10T10:14:23.491 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 285538231 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 4080698580 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-09-10T10:14:23.493 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2561106187 2024-09-10T10:14:23.494 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.494 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.494 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:23.494 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-09-10T10:14:23.494 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T10:14:23.495 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:22 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/134825144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:14:23.495 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:22 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2050791765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:14:23.495 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:22 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/4248329187' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:14:23.495 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:22 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/134825144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:14:23.495 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:22 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2050791765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:14:23.495 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:22 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/4248329187' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:14:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f55640a2d00 msgr2=0x7f55640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f55640a2d00 0x7f55640a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f556c005ab0 tx=0x7f556c005a20 comp rx=0 tx=0).stop 2024-09-10T10:14:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c19ccf0 msgr2=0x7f557c19d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c19ccf0 0x7f557c19d1a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5574009fd0 tx=0x7f557400b260 comp rx=0 tx=0).stop 2024-09-10T10:14:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 shutdown_connections 2024-09-10T10:14:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f557c19ccf0 0x7f557c19d1a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:23.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f55640a2d00 0x7f55640a51c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557c1080d0 0x7f557c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 --2- 172.21.15.45:0/4248329187 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f557c100880 0x7f557c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.245+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 >> 172.21.15.45:0/4248329187 conn(0x7f557c075f60 msgr2=0x7f557c105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.246+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 shutdown_connections 2024-09-10T10:14:23.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:22.246+0000 7f5582f31640 1 -- 172.21.15.45:0/4248329187 wait complete. 2024-09-10T10:14:23.548 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T10:14:23.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T10:14:23.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:22 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/134825144' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:14:23.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:22 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2050791765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:14:23.549 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:22 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/4248329187' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:14:23.576 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:23 smithi045 ceph-mon[31114]: pgmap v349: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 26 KiB/s wr, 3 op/s 2024-09-10T10:14:23.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:23 smithi162 ceph-mon[34963]: pgmap v349: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 26 KiB/s wr, 3 op/s 2024-09-10T10:14:23.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:23 smithi047 ceph-mon[35244]: pgmap v349: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 4.0 KiB/s rd, 26 KiB/s wr, 3 op/s 2024-09-10T10:14:23.819 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:24.397 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001305601, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000134342, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:24.398 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 224, 2024-09-10T10:14:24.399 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 212, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1445600, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 129632, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.047146201, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.017960084, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.041041116, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 212, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.023646843, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000111541 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.400 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1445600, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 129632 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 161, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 185472, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:24.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.016975741, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.011756808, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001697878, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 158, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.034372087, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000217544 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:24.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 185472 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:24.403 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-10T10:14:24.405 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148955, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-10T10:14:24.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 222, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1411595, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 222, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 222, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1411595, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:24.408 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:24.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:24.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:24.409 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T10:14:25.282 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-10T10:14:25.282 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-10T10:14:25.282 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-10T10:14:25.282 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T10:14:25.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T10:14:25.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:25 smithi162 ceph-mon[34963]: pgmap v350: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 6 op/s 2024-09-10T10:14:25.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:25 smithi045 ceph-mon[31114]: pgmap v350: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 6 op/s 2024-09-10T10:14:25.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:25 smithi047 ceph-mon[35244]: pgmap v350: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 6 op/s 2024-09-10T10:14:25.767 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2992, 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.003551675, 2024-09-10T10:14:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000050994, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002523641, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004423562, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001474520 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:14:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 226, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 216, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1444304, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 132544, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.052934037, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.017770947, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.050665306, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 215, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.040049326, 2024-09-10T10:14:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000186275 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1444304, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 132544 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 165, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 186176, 2024-09-10T10:14:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.018336727, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010721968, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002132508, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 162, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.042384932, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000261635 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 186176 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 149034, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 32, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-10T10:14:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 228, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1412439, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 228, 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 228, 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1412439, 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:26.438 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T10:14:27.108 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-10T10:14:27.109 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-10T10:14:27.109 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-10T10:14:27.109 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T10:14:27.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T10:14:27.367 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:27.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:27 smithi045 ceph-mon[31114]: pgmap v351: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 36 op/s 2024-09-10T10:14:27.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:27 smithi162 ceph-mon[34963]: pgmap v351: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 36 op/s 2024-09-10T10:14:27.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:27 smithi047 ceph-mon[35244]: pgmap v351: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 36 op/s 2024-09-10T10:14:27.897 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T10:14:27.897 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 223, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 213, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1443488, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 130624, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.054121469, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.017189284, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.049594304, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:27.898 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 212, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.026203285, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000123600 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1443488, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 130624 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:27.899 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 163, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 185824, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.016330120, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010900210, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002034812, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 160, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.037345016, 2024-09-10T10:14:27.900 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000233406 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 185824 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-09-10T10:14:27.901 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2992, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.002597992, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000041061, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002519756, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004564494, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001521498 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:27.902 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:27.903 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:27.904 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:27.905 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148994, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:27.906 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:27.907 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 225, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1412049, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 225, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 225, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1412049, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:27.909 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:27.909 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T10:14:28.771 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-10T10:14:28.771 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-10T10:14:28.771 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-10T10:14:28.771 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T10:14:28.771 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T10:14:29.030 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:29.475 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:29 smithi045 ceph-mon[31114]: pgmap v352: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 126 op/s 2024-09-10T10:14:29.497 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-09-10T10:14:29.497 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:29.497 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2992, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.003278752, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000054722, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002219009, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.004120962, 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001373654 2024-09-10T10:14:29.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 167, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 186528, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:29.499 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.016253715, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.010980305, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001985180, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 164, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.017197490, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000104862 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:29.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 186528 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 228, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 218, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1444848, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 133824, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.060480675, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.019258626, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.053807137, 2024-09-10T10:14:29.501 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 217, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.021835695, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000100625 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1444848, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 133824 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.502 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:29.503 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:29.504 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 150008, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:29.505 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:29.506 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 230, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1412699, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 230, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-09-10T10:14:29.507 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 230, 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1412699, 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-09-10T10:14:29.508 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-09-10T10:14:29.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:29 smithi162 ceph-mon[34963]: pgmap v352: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 126 op/s 2024-09-10T10:14:29.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:29 smithi047 ceph-mon[35244]: pgmap v352: 97 pgs: 97 active+clean; 21 GiB data, 67 GiB used, 1005 GiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 126 op/s 2024-09-10T10:14:30.303 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-10T10:14:30.303 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-10T10:14:30.303 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-10T10:14:30.303 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T10:14:30.303 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T10:14:30.578 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 346690, 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 348492, 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 120991914, 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 482321503, 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 20, 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 5, 2024-09-10T10:14:31.112 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 19.704009755, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 8.803985698, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 22.230721328, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.592212034, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 348491, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 45.624235923, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000130919 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 555878, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.168888496, 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000303 2024-09-10T10:14:31.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 336256, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 99680 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 4000, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 3988, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 2414383, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 303816185, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 13, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 6, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 1.662170177, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 1.169530450, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.113088995, 2024-09-10T10:14:31.114 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.371681064, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3987, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 1.174731091, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000294640 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 6, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000574, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000095 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1478944, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 139680 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.115 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2662, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2580, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 721140, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 287145331, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 87, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 7, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 1.255998858, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.895375674, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.099804842, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.241522188, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2576, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.961800333, 2024-09-10T10:14:31.116 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000373369 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 466, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000082192, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000176 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 140320, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 491296 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 1, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.117 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-MDSRank": { 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3529, 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 227.001661744, 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.064324642 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "finisher-PurgeQueue": { 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "queue_len": 0, 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "complete_latency": { 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2498, 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.081665777, 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000032692 2024-09-10T10:14:31.118 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": { 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "request": 219, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "reply": 219, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "reply_latency": { 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 219, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 10.038330838, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.045837127 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_reply": 0, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "forward": 0, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_complete": 0, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_fetch_keys": 0, 2024-09-10T10:14:31.119 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_commit": 131, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_split": 0, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_merge": 0, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes": 128, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_top": 45, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_bottom": 83, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pin_tail": 0, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_pinned": 63, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_expired": 0, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "inodes_with_caps": 3, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": 4, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "subtrees": 2, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse": 262, 2024-09-10T10:14:31.120 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_hit": 254, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_forward": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_discover": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_remote_ino": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "traverse_lock": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "load_cent": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "q": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "exported": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "exported_inodes": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "imported": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "imported_inodes": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_dir_fetch": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "openino_peer_discover": 0, 2024-09-10T10:14:31.121 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rfiles": 21, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rbytes": 10738134436, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "root_rsnaps": 6, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_set_tag": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_file_inodes": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_revoke": 47, 2024-09-10T10:14:31.122 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_grant": 3029, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps": 340670, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_caps_dirty": 340602, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_cap_release": 2, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "process_request_cap_release": 50 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_cache": { 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update": 0, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_update_receipt": 0, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_try_discover": 0, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_send_discover": 0, 2024-09-10T10:14:31.123 INFO:teuthology.orchestra.run.smithi047.stdout: "dir_handle_discover": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays": 4, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_delayed": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_created": 22, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_enqueued": 18, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_reintegrated": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "strays_migrated": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_processing": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_started": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "recovery_completed": 0, 2024-09-10T10:14:31.124 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_exportdir": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_flush": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_fragstats": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ireq_inodestats": 0 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_log": { 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "evlrg": 0, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "evadd": 341844, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "evex": 341780, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "evtrm": 341780, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "ev": 64, 2024-09-10T10:14:31.125 INFO:teuthology.orchestra.run.smithi047.stdout: "evexg": 0, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "evexd": 4, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "segadd": 488, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "segex": 487, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "segtrm": 487, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "seg": 1, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "segmjr": 1, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "segexg": 0, 2024-09-10T10:14:31.126 INFO:teuthology.orchestra.run.smithi047.stdout: "segexd": 1, 2024-09-10T10:14:31.127 INFO:teuthology.orchestra.run.smithi047.stdout: "expos": 1135041226, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "wrpos": 1135364479, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "rdpos": 0, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "jlat": { 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 6097, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 1231.209995218, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.201937017 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "replayed": 0 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_mem": { 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "ino": 100, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "ino+": 118, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "ino-": 18, 2024-09-10T10:14:31.128 INFO:teuthology.orchestra.run.smithi047.stdout: "dir": 22, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "dir+": 22, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "dir-": 0, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "dn": 128, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "dn+": 146, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "dn-": 18, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "cap": 4, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "cap+": 41, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "cap-": 37, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "rss": 79080, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "heap": 231712 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_server": { 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_client_request": 257, 2024-09-10T10:14:31.129 INFO:teuthology.orchestra.run.smithi047.stdout: "dispatch_server_request": 0, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_request": 219, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_client_session": 31, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "handle_peer_request": 0, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "req_create_latency": { 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 22, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.020242368, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000920107 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getattr_latency": { 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 115, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 2.072915544, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.018025352 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getfilelock_latency": { 2024-09-10T10:14:31.130 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "req_link_latency": { 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookup_latency": { 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.001004642, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000200928 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookuphash_latency": { 2024-09-10T10:14:31.131 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupino_latency": { 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000241580, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000120790 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupname_latency": { 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupparent_latency": { 2024-09-10T10:14:31.132 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lookupsnap_latency": { 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "req_lssnap_latency": { 2024-09-10T10:14:31.133 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 11, 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.001697713, 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000154337 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mkdir_latency": { 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 10, 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.011950398, 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.001195039 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mknod_latency": { 2024-09-10T10:14:31.134 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "req_mksnap_latency": { 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 11, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 7.237516997, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.657956090 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "req_open_latency": { 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000161443, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000161443 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.135 INFO:teuthology.orchestra.run.smithi047.stdout: "req_readdir_latency": { 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.013565272, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.002713054 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rename_latency": { 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 2, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.001890604, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000945302 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "req_renamesnap_latency": { 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.136 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "req_snapdiff_latency": { 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmdir_latency": { 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 4, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.602652090, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.150663022 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmsnap_latency": { 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 5, 2024-09-10T10:14:31.137 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.029489040, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.005897808 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "req_rmxattr_latency": { 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setattr_latency": { 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 1, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.001157578, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.001157578 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.138 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setdirlayout_latency": { 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setfilelock_latency": { 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setlayout_latency": { 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.139 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "req_setxattr_latency": { 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 7, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.019856766, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.002836680 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "req_symlink_latency": { 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "req_unlink_latency": { 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 18, 2024-09-10T10:14:31.140 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.023988803, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.001332711 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "req_getvxattr_latency": { 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_sessions": { 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "session_count": 2, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "session_add": 5, 2024-09-10T10:14:31.141 INFO:teuthology.orchestra.run.smithi047.stdout: "session_remove": 3, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_open": 2, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "sessions_stale": 0, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "total_load": 49, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "average_load": 24, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "avg_session_uptime": 654, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "mdthresh_evicted": 0 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:31.142 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:31.143 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:31.144 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 356053, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 162, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 528, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 6, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 8192, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 180, 2024-09-10T10:14:31.145 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 379017, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2765, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "objecter": { 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "op_active": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "op_laggy": 0, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send": 10482, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "op_send_bytes": 1131932639, 2024-09-10T10:14:31.146 INFO:teuthology.orchestra.run.smithi047.stdout: "op_resend": 32, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "op_reply": 10482, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "op_latency": { 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 10482, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 643.174200190, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.061359874 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "op_inflight": 0, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "oplen_avg": { 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 10482, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 10829 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "op": 10482, 2024-09-10T10:14:31.147 INFO:teuthology.orchestra.run.smithi047.stdout: "op_r": 1, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "op_w": 10481, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "op_rmw": 0, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "op_pg": 0, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_stat": 109, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_create": 44, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_read": 1, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_write": 6098, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writefull": 208, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_writesame": 0, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_append": 0, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_zero": 0, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_truncate": 0, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_delete": 3628, 2024-09-10T10:14:31.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_mapext": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_sparse_read": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_clonerange": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_getxattr": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_setxattr": 88, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_rmxattr": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_call": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_watch": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_notify": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdop_other": 0, 2024-09-10T10:14:31.149 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_active": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_send": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_resend": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "linger_ping": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_active": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_send": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "poolop_resend": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_active": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_send": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "poolstat_resend": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_active": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_send": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "statfs_resend": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "command_active": 0, 2024-09-10T10:14:31.150 INFO:teuthology.orchestra.run.smithi047.stdout: "command_send": 0, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "command_resend": 0, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "map_epoch": 95, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "map_full": 0, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "map_inc": 14, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_sessions": 12, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_open": 12, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_session_close": 0, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_laggy": 0, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_wr": 637, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_rd": 0, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_del": 16 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.151 INFO:teuthology.orchestra.run.smithi047.stdout: "oft": { 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_objs": 1, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_updates": 0, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "omap_total_removes": 0 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "osd-slow-ops": { 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "slow_ops_count": 0 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "purge_queue": { 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops": 0, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_ops_high_water": 519, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing": 0, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executing_high_water": 7, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed_ops": 2578, 2024-09-10T10:14:31.152 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_executed": 18, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "pq_item_in_journal": 0 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 353348, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 92259293, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 353348, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 353348, 2024-09-10T10:14:31.153 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 92259293, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_bytes": { 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:31.154 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 10482, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 1131935321, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 10482, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1131935321, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-objecter_ops": { 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 1024, 2024-09-10T10:14:31.155 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 0, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 0, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 0, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 10482, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 10482, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 10482, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 10482, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 18, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 2682, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 18, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 1, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 2682, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.157 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-write_buf_throttle-0x559f8cb84960": { 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 3758096384, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 341844, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1131170175, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 341844, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 6097, 2024-09-10T10:14:31.158 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1131170175, 2024-09-10T10:14:31.159 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:31.159 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:31.159 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:31.159 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:31.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.159 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:31.159 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-10T10:14:31.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:31 smithi162 ceph-mon[34963]: pgmap v353: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 202 op/s 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:31 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:31 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:31 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:31 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:31 smithi045 ceph-mon[31114]: pgmap v353: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 202 op/s 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:31 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:31 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:14:31.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:31 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:14:31.651 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:31 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:14:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:31 smithi047 ceph-mon[35244]: pgmap v353: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 202 op/s 2024-09-10T10:14:31.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:31 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:14:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:31 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:14:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:31 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:14:31.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:31 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' 2024-09-10T10:14:31.962 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-10T10:14:31.962 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-10T10:14:31.963 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-10T10:14:31.963 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T10:14:31.963 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T10:14:32.217 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.001224837, 2024-09-10T10:14:32.692 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000261990, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.693 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 165, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186176, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.011639213, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.007492990, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001871632, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:32.694 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 162, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.020619935, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000127283 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186176 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 231, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 219, 2024-09-10T10:14:32.695 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1447568, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 133632, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.042647959, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.014382623, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.040639495, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 219, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.018223516, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000083212 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.696 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1447568, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 133632 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:32.697 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:32.699 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 150025, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 32, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:32.700 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:32.701 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 229, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1412569, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 229, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 229, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1412569, 2024-09-10T10:14:32.702 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-10T10:14:33.361 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-10T10:14:33.361 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-10T10:14:33.361 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-10T10:14:33.361 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T10:14:33.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T10:14:33.614 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:14:33.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:33 smithi162 ceph-mon[34963]: pgmap v354: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 KiB/s wr, 220 op/s 2024-09-10T10:14:33.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:33 smithi045 ceph-mon[31114]: pgmap v354: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 KiB/s wr, 220 op/s 2024-09-10T10:14:33.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:33 smithi047 ceph-mon[35244]: pgmap v354: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 26 KiB/s rd, 5.7 KiB/s wr, 220 op/s 2024-09-10T10:14:34.139 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 230, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 218, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1447296, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 132992, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.043864155, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.014653522, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.040490339, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:34.140 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 218, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.020667140, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000094803 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1447296, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 132992 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.141 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 165, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186176, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.010694091, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.006878700, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.001774171, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 162, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.016856115, 2024-09-10T10:14:34.142 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000104050 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186176 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 0, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 2, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:14:34.143 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 0, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.000341072, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.000262875, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:34.144 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:34.145 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:34.146 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:34.147 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148994, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 31, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:34.148 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-09-10T10:14:34.149 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 228, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1412439, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 228, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-10T10:14:34.150 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 228, 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1412439, 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:34.151 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-10T10:14:34.821 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-09-10T10:14:34.821 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-09-10T10:14:34.822 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-09-10T10:14:34.822 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T10:14:34.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T10:14:35.074 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:14:35.512 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:35 smithi047 ceph-mon[35244]: pgmap v355: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 KiB/s wr, 222 op/s 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 4, 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 4, 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 2992, 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 448, 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:35.539 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.003119889, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.000037131, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.002243703, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 3, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.004056430, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.001352143 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.540 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 168, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 186704, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.012910101, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.008039961, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.002577477, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:35.541 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 165, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.018894969, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000114514 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 186704 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.542 INFO:teuthology.orchestra.run.smithi047.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_messages": 230, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages": 220, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_bytes": 1445392, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_bytes": 135104, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_total_time": 0.042035291, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_send_time": 0.014596132, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_recv_time": 0.040674788, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 219, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.011800957, 2024-09-10T10:14:35.543 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000053885 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_recv_encrypted_bytes": 1445392, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "msgr_send_encrypted_bytes": 135104 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "cct": { 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "total_workers": 0, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "unhealthy_workers": 0 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.544 INFO:teuthology.orchestra.run.smithi047.stdout: "mempool": { 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:35.545 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:35.546 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_items": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_bytes": 148977, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_anon_items": 30, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_bytes": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_items": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:35.547 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_items": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "pgmap_items": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_co_items": 2, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_1_items": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "unittest_2_items": 0 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "val": 0, 2024-09-10T10:14:35.548 INFO:teuthology.orchestra.run.smithi047.stdout: "max": 104857600, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "get_started": 0, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "get": 232, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "get_sum": 1412959, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "get_or_fail_success": 232, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "take": 0, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "take_sum": 0, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "put": 232, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "put_sum": 1412959, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "wait": { 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "avgcount": 0, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "sum": 0.000000000, 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: "avgtime": 0.000000000 2024-09-10T10:14:35.549 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:35.550 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-09-10T10:14:35.550 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-09-10T10:14:35.611 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:35 smithi045 ceph-mon[31114]: pgmap v355: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 KiB/s wr, 222 op/s 2024-09-10T10:14:35.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:35 smithi162 ceph-mon[34963]: pgmap v355: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 29 KiB/s rd, 8.7 KiB/s wr, 222 op/s 2024-09-10T10:14:36.167 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-10T10:14:36.168 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-10T10:14:36.168 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-10T10:14:36.168 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T10:14:36.168 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T10:14:36.429 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:14:36.882 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 0, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 0, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.001349133, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000338473, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:36.883 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 165, 2024-09-10T10:14:36.884 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 186176, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.011444185, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.007335150, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.001897186, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 162, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.014482804, 2024-09-10T10:14:36.885 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000089400 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 186176 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 231, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 219, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1447568, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 133632, 2024-09-10T10:14:36.886 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.047927515, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.015547552, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.045530700, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 219, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.022070017, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000100776 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:36.887 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1447568, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 133632 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:36.888 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:36.889 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:36.890 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148977, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 30, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:36.891 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 229, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1412569, 2024-09-10T10:14:36.892 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 229, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 229, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1412569, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:36.893 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-10T10:14:37.579 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:37 smithi162 ceph-mon[34963]: pgmap v356: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.6 KiB/s wr, 217 op/s 2024-09-10T10:14:37.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:37 smithi045 ceph-mon[31114]: pgmap v356: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.6 KiB/s wr, 217 op/s 2024-09-10T10:14:37.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:37 smithi047 ceph-mon[35244]: pgmap v356: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.6 KiB/s wr, 217 op/s 2024-09-10T10:14:37.673 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-09-10T10:14:37.674 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-09-10T10:14:37.674 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-09-10T10:14:37.674 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T10:14:37.674 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T10:14:37.922 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:14:38.443 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-10T10:14:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 0, 2024-09-10T10:14:38.443 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 2, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 0, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.001104814, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.000211062, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:38.444 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 6, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 169, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 3312, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 186976, 2024-09-10T10:14:38.445 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.013415584, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.007665040, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.003578777, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 167, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.018814417, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000112661 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:38.446 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 186976 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 228, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 218, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1444848, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 133824, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:38.447 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.045798200, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.015476451, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.043941839, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 217, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.012591010, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000058023 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:38.448 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1444848, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 133824 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:38.449 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:38.450 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:38.451 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148977, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 30, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:38.452 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-09-10T10:14:38.453 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 230, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1412699, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 230, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 230, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1412699, 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:38.454 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:38.455 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:38.455 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-10T10:14:39.079 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-09-10T10:14:39.079 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-09-10T10:14:39.079 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-09-10T10:14:39.079 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T10:14:39.079 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T10:14:39.337 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:14:39.578 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:39 smithi162 ceph-mon[34963]: pgmap v357: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 193 op/s 2024-09-10T10:14:39.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:39 smithi045 ceph-mon[31114]: pgmap v357: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 193 op/s 2024-09-10T10:14:39.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:39 smithi047 ceph-mon[35244]: pgmap v357: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 193 op/s 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 707, 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 704, 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 88469536, 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 252336, 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 7, 2024-09-10T10:14:39.799 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.146910864, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.025128920, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.095371312, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.022806278, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 704, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.047464943, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000067421 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.800 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 192400, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 68528 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 373, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 453, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 192357506, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 344801, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 7, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 6, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.188753931, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.012439715, 2024-09-10T10:14:39.801 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.155498748, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.014352911, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 451, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.156700562, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000347451 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 226448 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.802 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 1027, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 1029, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 225502074, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 326602, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 6, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 5, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.231635919, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.026367885, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.166455752, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.030356605, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1027, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.049313700, 2024-09-10T10:14:39.803 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000048017 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 1, 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-09-10T10:14:39.804 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-MDSRank": { 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 2847, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.090383423, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000031746 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "finisher-PurgeQueue": { 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "queue_len": 0, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "complete_latency": { 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.805 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "mds": { 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "request": 0, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "reply": 0, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "reply_latency": { 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_reply": 0, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "forward": 0, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_complete": 0, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_fetch_keys": 0, 2024-09-10T10:14:39.806 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_commit": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_split": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_merge": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes": 94, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_top": 16, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_bottom": 78, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pin_tail": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_pinned": 71, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_expired": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "inodes_with_caps": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "caps": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "subtrees": 2, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_hit": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_forward": 0, 2024-09-10T10:14:39.807 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_discover": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_remote_ino": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "traverse_lock": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "load_cent": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "q": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "exported": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "exported_inodes": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "imported": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "imported_inodes": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_dir_fetch": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "openino_peer_discover": 0, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rfiles": 4, 2024-09-10T10:14:39.808 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rbytes": 234, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "root_rsnaps": 6, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_set_tag": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "scrub_file_inodes": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:14:39.809 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_cap_release": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "process_request_cap_release": 0 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_cache": { 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_update_receipt": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_try_discover": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_send_discover": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "dir_handle_discover": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_delayed": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_created": 0, 2024-09-10T10:14:39.810 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_enqueued": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_reintegrated": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "strays_migrated": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_processing": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_started": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "recovery_completed": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_exportdir": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_flush": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_fragstats": 0, 2024-09-10T10:14:39.811 INFO:teuthology.orchestra.run.smithi162.stdout: "ireq_inodestats": 0 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_log": { 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "evlrg": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "evadd": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "evex": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "evtrm": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "ev": 113670, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "evexg": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "evexd": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "segadd": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "segex": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "segtrm": 0, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "seg": 2, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "segmjr": 33, 2024-09-10T10:14:39.812 INFO:teuthology.orchestra.run.smithi162.stdout: "segexg": 0, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "segexd": 0, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "expos": 1135041226, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "wrpos": 1135366094, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "rdpos": 1135364479, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "jlat": { 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "replayed": 113670 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_mem": { 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "ino": 53, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "ino+": 71, 2024-09-10T10:14:39.813 INFO:teuthology.orchestra.run.smithi162.stdout: "ino-": 18, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dir": 11, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dir+": 11, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dir-": 0, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dn": 94, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dn+": 94, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dn-": 0, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "cap": 0, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "cap+": 0, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "cap-": 0, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "rss": 50000, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "heap": 215328 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_server": { 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_client_request": 0, 2024-09-10T10:14:39.814 INFO:teuthology.orchestra.run.smithi162.stdout: "dispatch_server_request": 0, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_request": 0, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_client_session": 0, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "handle_peer_request": 0, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "req_create_latency": { 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getattr_latency": { 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getfilelock_latency": { 2024-09-10T10:14:39.815 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "req_link_latency": { 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookup_latency": { 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookuphash_latency": { 2024-09-10T10:14:39.816 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupino_latency": { 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupname_latency": { 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupparent_latency": { 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.817 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lookupsnap_latency": { 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "req_lssnap_latency": { 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.818 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mkdir_latency": { 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mknod_latency": { 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.819 INFO:teuthology.orchestra.run.smithi162.stdout: "req_mksnap_latency": { 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "req_open_latency": { 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "req_readdir_latency": { 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.820 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rename_latency": { 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "req_renamesnap_latency": { 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "req_snapdiff_latency": { 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.821 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmdir_latency": { 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmsnap_latency": { 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "req_rmxattr_latency": { 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.822 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setattr_latency": { 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setdirlayout_latency": { 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setfilelock_latency": { 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.823 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setlayout_latency": { 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "req_setxattr_latency": { 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "req_symlink_latency": { 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.824 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "req_unlink_latency": { 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "req_getvxattr_latency": { 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.825 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_sessions": { 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "session_count": 2, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "session_add": 2, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "session_remove": 0, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_open": 0, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "sessions_stale": 0, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "total_load": 0, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "average_load": 0, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "avg_session_uptime": 1938, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "mdthresh_evicted": 0 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:39.826 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:39.827 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:39.828 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 239902, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 131, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:39.829 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 8192, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 180, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 240308, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2099, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.830 INFO:teuthology.orchestra.run.smithi162.stdout: "objecter": { 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_active": 0, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_laggy": 0, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send": 1977, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_send_bytes": 26, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_resend": 4, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_reply": 1977, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_latency": { 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1977, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 13.090738851, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.006621516 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "op_inflight": 0, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "oplen_avg": { 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 1977, 2024-09-10T10:14:39.831 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 1978 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "op": 1977, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "op_r": 1977, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "op_w": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "op_rmw": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "op_pg": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_stat": 1349, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_create": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_read": 625, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_write": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writefull": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_writesame": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_append": 0, 2024-09-10T10:14:39.832 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_zero": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_truncate": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_delete": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_mapext": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_sparse_read": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_clonerange": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_getxattr": 2, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_setxattr": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_rmxattr": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_call": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_watch": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_notify": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:14:39.833 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "osdop_other": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_active": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_send": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_resend": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "linger_ping": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_active": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_send": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "poolop_resend": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_active": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_send": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "poolstat_resend": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_active": 0, 2024-09-10T10:14:39.834 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_send": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "statfs_resend": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "command_active": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "command_send": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "command_resend": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "map_epoch": 95, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "map_full": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "map_inc": 10, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_sessions": 11, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_open": 11, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_session_close": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_laggy": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_wr": 0, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_rd": 2, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_del": 0 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.835 INFO:teuthology.orchestra.run.smithi162.stdout: "oft": { 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_objs": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_updates": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "omap_total_removes": 0 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "osd-slow-ops": { 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "slow_ops_count": 0 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "purge_queue": { 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executing_high_water": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed_ops": 0, 2024-09-10T10:14:39.836 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_executed": 0, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "pq_item_in_journal": 0 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 2103, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 506130487, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 2103, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 2103, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 506130487, 2024-09-10T10:14:39.837 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_bytes": { 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 1977, 2024-09-10T10:14:39.838 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 505595612, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 1977, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 505595612, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-objecter_ops": { 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 1024, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-09-10T10:14:39.839 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 1977, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 1977, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 1977, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1977, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:39.840 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.841 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-write_buf_throttle-0x564fdfc6c6e0": { 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 3758096384, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:39.842 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:39.843 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:39.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.843 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:39.843 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-10T10:14:40.616 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-10T10:14:40.616 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-10T10:14:40.616 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T10:14:40.616 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T10:14:40.617 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T10:14:40.866 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:14:41.337 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-10T10:14:41.337 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:14:41.337 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 4, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 4, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 2992, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 448, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 4, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 0, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.003055764, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.000038278, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.002172730, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 3, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.004009888, 2024-09-10T10:14:41.338 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.001336629 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 232, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 222, 2024-09-10T10:14:41.339 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 1445936, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 136384, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 2, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 1, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.042812823, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.015629702, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.041362285, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 221, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.014037330, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000063517 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.340 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 1445936, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 136384 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_messages": 2, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages": 170, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_bytes": 187056, 2024-09-10T10:14:41.341 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_created_connections": 5, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_active_connections": 3, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_total_time": 0.012144994, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_send_time": 0.008197805, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_recv_time": 0.001621869, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 167, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.009835437, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000058894 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:41.342 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "msgr_send_encrypted_bytes": 187056 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "cct": { 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "total_workers": 0, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "unhealthy_workers": 0 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "mempool": { 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "bloom_filter_items": 0, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:14:41.343 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_extent_items": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:14:41.344 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_blob_items": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_txc_items": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluestore_writing_items": 0, 2024-09-10T10:14:41.345 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_bytes": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_items": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_bytes": 148994, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_anon_items": 31, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "buffer_meta_items": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_bytes": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_items": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_mapbl_items": 0, 2024-09-10T10:14:41.346 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "osd_pglog_items": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_bytes": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_items": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_bytes": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "pgmap_items": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_bytes": 928, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "mds_co_items": 2, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_bytes": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_1_items": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_bytes": 0, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "unittest_2_items": 0 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: }, 2024-09-10T10:14:41.347 INFO:teuthology.orchestra.run.smithi162.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "val": 0, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "max": 104857600, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "get_started": 0, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "get": 234, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "get_sum": 1413219, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_fail": 0, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "get_or_fail_success": 234, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "take": 0, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "take_sum": 0, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "put": 234, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "put_sum": 1413219, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "wait": { 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "avgcount": 0, 2024-09-10T10:14:41.348 INFO:teuthology.orchestra.run.smithi162.stdout: "sum": 0.000000000, 2024-09-10T10:14:41.349 INFO:teuthology.orchestra.run.smithi162.stdout: "avgtime": 0.000000000 2024-09-10T10:14:41.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:41.349 INFO:teuthology.orchestra.run.smithi162.stdout: } 2024-09-10T10:14:41.349 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-10T10:14:41.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:41 smithi162 ceph-mon[34963]: pgmap v358: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 100 op/s 2024-09-10T10:14:41.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:41 smithi045 ceph-mon[31114]: pgmap v358: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 100 op/s 2024-09-10T10:14:41.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:41 smithi047 ceph-mon[35244]: pgmap v358: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 100 op/s 2024-09-10T10:14:42.016 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-10T10:14:42.016 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-10T10:14:42.016 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-10T10:14:42.016 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T10:14:42.026 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T10:14:42.026 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T10:14:42.026 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:14:42.047 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-09-10T10:14:42.047 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:14:42.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T10:14:42.417 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 -- 172.21.15.45:0/988247722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9041027a0 msgr2=0x7fe9041067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 --2- 172.21.15.45:0/988247722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9041027a0 0x7fe9041067f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009900 tx=0x7fe8f402a410 comp rx=0 tx=0).stop 2024-09-10T10:14:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 -- 172.21.15.45:0/988247722 shutdown_connections 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 --2- 172.21.15.45:0/988247722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe904106f20 0x7fe90410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 --2- 172.21.15.45:0/988247722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9041027a0 0x7fe9041067f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 --2- 172.21.15.45:0/988247722 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe904101df0 0x7fe9041021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 -- 172.21.15.45:0/988247722 >> 172.21.15.45:0/988247722 conn(0x7fe9040fd8b0 msgr2=0x7fe9040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 -- 172.21.15.45:0/988247722 shutdown_connections 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.949+0000 7fe90c65e640 1 -- 172.21.15.45:0/988247722 wait complete. 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 Processor -- start 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 -- start start 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe904101df0 0x7fe9040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9041027a0 0x7fe904074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe904106f20 0x7fe904074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe904111f80 con 0x7fe904106f20 2024-09-10T10:14:42.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe904111e00 con 0x7fe9041027a0 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90c65e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fe904112100 con 0x7fe904101df0 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe904106f20 0x7fe904074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe904106f20 0x7fe904074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52312/0 (socket says 172.21.15.45:52312) 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 -- 172.21.15.45:0/2316402751 learned_addr learned my addr 172.21.15.45:0/2316402751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe909bf9640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9041027a0 0x7fe904074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90a3fa640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe904101df0 0x7fe9040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 -- 172.21.15.45:0/2316402751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe904101df0 msgr2=0x7fe9040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe904101df0 0x7fe9040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 -- 172.21.15.45:0/2316402751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9041027a0 msgr2=0x7fe904074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9041027a0 0x7fe904074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.950+0000 7fe90abfb640 1 -- 172.21.15.45:0/2316402751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9041a6930 con 0x7fe904106f20 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.951+0000 7fe90abfb640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe904106f20 0x7fe904074f70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00eb10 tx=0x7fe8fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:42.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.951+0000 7fe8fb7fe640 1 -- 172.21.15.45:0/2316402751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc004b20 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.951+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9041a6c20 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.951+0000 7fe8fb7fe640 1 -- 172.21.15.45:0/2316402751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8fc004cc0 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.951+0000 7fe8fb7fe640 1 -- 172.21.15.45:0/2316402751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc010760 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.951+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9041a7130 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.952+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9041021d0 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.953+0000 7fe8fb7fe640 1 -- 172.21.15.45:0/2316402751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe8fc005020 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.954+0000 7fe8fb7fe640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe8f00a2cf0 0x7fe8f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.954+0000 7fe90a3fa640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe8f00a2cf0 0x7fe8f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.954+0000 7fe8fb7fe640 1 -- 172.21.15.45:0/2316402751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7fe8fc0cf960 con 0x7fe904106f20 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.954+0000 7fe90a3fa640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe8f00a2cf0 0x7fe8f00a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe900009bd0 tx=0x7fe900005f90 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:42.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:42.958+0000 7fe8fb7fe640 1 -- 172.21.15.45:0/2316402751 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8fc09c560 con 0x7fe904106f20 2024-09-10T10:14:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.144+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe90410a160 con 0x7fe904106f20 2024-09-10T10:14:43.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.145+0000 7fe8fb7fe640 1 -- 172.21.15.45:0/2316402751 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v95) ==== 81+0+994 (secure 0 0 0) 0x7fe8fc09be60 con 0x7fe904106f20 2024-09-10T10:14:43.146 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/952575454 2024-09-11T10:03:11.647760+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/952575454 2024-09-11T10:03:11.647760+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3649460339 2024-09-11T10:03:11.647760+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1519065487 2024-09-11T09:54:24.047197+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3534138672 2024-09-11T09:54:24.047197+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1473601885 2024-09-11T09:54:24.047197+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/444500118 2024-09-11T09:54:48.533581+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/986599852 2024-09-11T09:54:24.047197+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/384122965 2024-09-11T10:03:11.647760+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/986599852 2024-09-11T09:54:24.047197+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/689699829 2024-09-11T10:03:11.647760+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2521990938 2024-09-11T09:54:48.533581+0000 2024-09-10T10:14:43.147 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1958743370 2024-09-11T09:54:48.533581+0000 2024-09-10T10:14:43.148 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/375260435 2024-09-11T10:03:11.647760+0000 2024-09-10T10:14:43.148 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1958743370 2024-09-11T09:54:48.533581+0000 2024-09-10T10:14:43.148 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/4168396686 2024-09-11T09:54:48.533581+0000 2024-09-10T10:14:43.148 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/21896051 2024-09-11T10:03:11.647760+0000 2024-09-10T10:14:43.148 INFO:teuthology.orchestra.run.smithi045.stderr:listed 17 entries 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe8f00a2cf0 msgr2=0x7fe8f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe8f00a2cf0 0x7fe8f00a51b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe900009bd0 tx=0x7fe900005f90 comp rx=0 tx=0).stop 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe904106f20 msgr2=0x7fe904074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe904106f20 0x7fe904074f70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00eb10 tx=0x7fe8fc00c2d0 comp rx=0 tx=0).stop 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 shutdown_connections 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe904106f20 0x7fe904074f70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fe8f00a2cf0 0x7fe8f00a51b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9041027a0 0x7fe904074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 --2- 172.21.15.45:0/2316402751 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe904101df0 0x7fe9040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 >> 172.21.15.45:0/2316402751 conn(0x7fe9040fd8b0 msgr2=0x7fe9040feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 shutdown_connections 2024-09-10T10:14:43.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:43.147+0000 7fe90c65e640 1 -- 172.21.15.45:0/2316402751 wait complete. 2024-09-10T10:14:43.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:43 smithi045 ceph-mon[31114]: pgmap v359: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-09-10T10:14:43.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:43 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2316402751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:14:43.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:43 smithi162 ceph-mon[34963]: pgmap v359: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-09-10T10:14:43.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:43 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2316402751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:14:43.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:43 smithi047 ceph-mon[35244]: pgmap v359: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.2 KiB/s wr, 21 op/s 2024-09-10T10:14:43.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:43 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2316402751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:14:43.761 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:14:43.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:14:43.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:14:43.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:14:43.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:14:43.805 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:14:43.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:14:43.893 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-09-10T10:14:43.916 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:14:43.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:14:43.917 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:14:43.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:14:43.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-09-10T10:14:44.069 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T10:14:44.082 INFO:tasks.cephadm:Teardown begin 2024-09-10T10:14:44.082 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:14:44.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:14:44.369 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 -- 172.21.15.45:0/2635471095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec410c170 msgr2=0x7faec410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 --2- 172.21.15.45:0/2635471095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec410c170 0x7faec410e610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faebc009ca0 tx=0x7faebc02a950 comp rx=0 tx=0).stop 2024-09-10T10:14:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 -- 172.21.15.45:0/2635471095 shutdown_connections 2024-09-10T10:14:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 --2- 172.21.15.45:0/2635471095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec410c170 0x7faec410e610 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 --2- 172.21.15.45:0/2635471095 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faec4103660 0x7faec410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 --2- 172.21.15.45:0/2635471095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec4102d40 0x7faec4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 -- 172.21.15.45:0/2635471095 >> 172.21.15.45:0/2635471095 conn(0x7faec40fc7a0 msgr2=0x7faec40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 -- 172.21.15.45:0/2635471095 shutdown_connections 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 -- 172.21.15.45:0/2635471095 wait complete. 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.856+0000 7faeca253640 1 Processor -- start 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faeca253640 1 -- start start 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faeca253640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec4102d40 0x7faec41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faeca253640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec4103660 0x7faec4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faeca253640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faec410c170 0x7faec4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faeca253640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faec4111fa0 con 0x7faec4102d40 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faeca253640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faec4111e20 con 0x7faec4103660 2024-09-10T10:14:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faeca253640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7faec4112120 con 0x7faec410c170 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec4102d40 0x7faec41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec4102d40 0x7faec41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55864/0 (socket says 172.21.15.45:55864) 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 -- 172.21.15.45:0/2780341738 learned_addr learned my addr 172.21.15.45:0/2780341738 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec3fff640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faec410c170 0x7faec4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec2ffd640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec4103660 0x7faec4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 -- 172.21.15.45:0/2780341738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faec410c170 msgr2=0x7faec4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faec410c170 0x7faec4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 -- 172.21.15.45:0/2780341738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec4103660 msgr2=0x7faec4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec4103660 0x7faec4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.857+0000 7faec37fe640 1 -- 172.21.15.45:0/2780341738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec41a6890 con 0x7faec4102d40 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.858+0000 7faec37fe640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec4102d40 0x7faec41009a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c890 tx=0x7faeb400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.858+0000 7faec0ff9640 1 -- 172.21.15.45:0/2780341738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4004220 con 0x7faec4102d40 2024-09-10T10:14:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.858+0000 7faec0ff9640 1 -- 172.21.15.45:0/2780341738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb40043c0 con 0x7faec4102d40 2024-09-10T10:14:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.858+0000 7faec0ff9640 1 -- 172.21.15.45:0/2780341738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb40049e0 con 0x7faec4102d40 2024-09-10T10:14:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.858+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec41a6b80 con 0x7faec4102d40 2024-09-10T10:14:44.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.858+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec41a70c0 con 0x7faec4102d40 2024-09-10T10:14:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.859+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec4103120 con 0x7faec4102d40 2024-09-10T10:14:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.860+0000 7faec0ff9640 1 -- 172.21.15.45:0/2780341738 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faeb40079b0 con 0x7faec4102d40 2024-09-10T10:14:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.861+0000 7faec0ff9640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faeac0a2cf0 0x7faeac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.861+0000 7faec2ffd640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faeac0a2cf0 0x7faeac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.861+0000 7faec0ff9640 1 -- 172.21.15.45:0/2780341738 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7faeb40cf420 con 0x7faec4102d40 2024-09-10T10:14:44.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.862+0000 7faec2ffd640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faeac0a2cf0 0x7faeac0a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faec4102430 tx=0x7faeb800b2e0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:44.865+0000 7faec0ff9640 1 -- 172.21.15.45:0/2780341738 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faeb4014030 con 0x7faec4102d40 2024-09-10T10:14:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.075+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faec41a7290 con 0x7faec4102d40 2024-09-10T10:14:45.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.076+0000 7faec0ff9640 1 -- 172.21.15.45:0/2780341738 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11843 (secure 0 0 0) 0x7faeb409b9d0 con 0x7faec4102d40 2024-09-10T10:14:45.077 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:14:45.077 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-10T10:08:10:809804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:08:10.809787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25127":{"gid":25127,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2699160150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2699160150},{"type":"v1","addr":"172.21.15.162:6839","nonce":2699160150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:14:45.078 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-10T10:14:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faeac0a2cf0 msgr2=0x7faeac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faeac0a2cf0 0x7faeac0a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faec4102430 tx=0x7faeb800b2e0 comp rx=0 tx=0).stop 2024-09-10T10:14:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec4102d40 msgr2=0x7faec41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:45.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec4102d40 0x7faec41009a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faeb400c890 tx=0x7faeb400cd50 comp rx=0 tx=0).stop 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 shutdown_connections 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7faec410c170 0x7faec4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7faeac0a2cf0 0x7faeac0a51b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faec4103660 0x7faec4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 --2- 172.21.15.45:0/2780341738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faec4102d40 0x7faec41009a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 >> 172.21.15.45:0/2780341738 conn(0x7faec40fc7a0 msgr2=0x7faec40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 shutdown_connections 2024-09-10T10:14:45.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:45.078+0000 7faeca253640 1 -- 172.21.15.45:0/2780341738 wait complete. 2024-09-10T10:14:45.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:45 smithi045 ceph-mon[31114]: pgmap v360: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-10T10:14:45.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:45 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2780341738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:14:45.649 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:14:45.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:45 smithi162 ceph-mon[34963]: pgmap v360: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-10T10:14:45.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:45 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2780341738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:14:45.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:45 smithi047 ceph-mon[35244]: pgmap v360: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-09-10T10:14:45.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:45 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2780341738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:14:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:46.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/989985377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4102570 msgr2=0x7f5df41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/989985377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4102570 0x7f5df41029b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009900 tx=0x7f5de802a480 comp rx=0 tx=0).stop 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/989985377 shutdown_connections 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/989985377 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df4102ef0 0x7f5df410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/989985377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4102570 0x7f5df41029b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/989985377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5df4100e20 0x7f5df4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/989985377 >> 172.21.15.45:0/989985377 conn(0x7f5df40fc7d0 msgr2=0x7f5df40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/989985377 shutdown_connections 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/989985377 wait complete. 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 Processor -- start 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.355+0000 7f5dfb0cf640 1 -- start start 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5dfb0cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4100e20 0x7f5df4073b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:46.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5dfb0cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df4102570 0x7f5df406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5dfb0cf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5df4102ef0 0x7f5df406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5dfb0cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4110fa0 con 0x7f5df4100e20 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5dfb0cf640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4110e20 con 0x7f5df4102ef0 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5dfb0cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4111120 con 0x7f5df4102570 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4100e20 0x7f5df4073b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4100e20 0x7f5df4073b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55878/0 (socket says 172.21.15.45:55878) 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 -- 172.21.15.45:0/1003345561 learned_addr learned my addr 172.21.15.45:0/1003345561 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df966c640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5df4102ef0 0x7f5df406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df3fff640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df4102570 0x7f5df406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 -- 172.21.15.45:0/1003345561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df4102570 msgr2=0x7f5df406eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df4102570 0x7f5df406eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 -- 172.21.15.45:0/1003345561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5df4102ef0 msgr2=0x7f5df406f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:46.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5df4102ef0 0x7f5df406f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 -- 172.21.15.45:0/1003345561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df406f6b0 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df966c640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5df4102ef0 0x7f5df406f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.356+0000 7f5df8e6b640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4100e20 0x7f5df4073b80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5de400e9f0 tx=0x7f5de400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.357+0000 7f5df1ffb640 1 -- 172.21.15.45:0/1003345561 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de400cd80 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.357+0000 7f5df1ffb640 1 -- 172.21.15.45:0/1003345561 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4010910 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.357+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df41ad4b0 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.357+0000 7f5df1ffb640 1 -- 172.21.15.45:0/1003345561 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4018990 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.357+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df41ad850 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.358+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df41021b0 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.360+0000 7f5df1ffb640 1 -- 172.21.15.45:0/1003345561 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5de4010ac0 con 0x7f5df4100e20 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.360+0000 7f5df1ffb640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5ddc0a2cf0 0x7f5ddc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.361+0000 7f5df3fff640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5ddc0a2cf0 0x7f5ddc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:46.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.361+0000 7f5df1ffb640 1 -- 172.21.15.45:0/1003345561 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f5de40cfcc0 con 0x7f5df4100e20 2024-09-10T10:14:46.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.361+0000 7f5df3fff640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5ddc0a2cf0 0x7f5ddc0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5df406aac0 tx=0x7f5de8005a20 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:46.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.365+0000 7f5df1ffb640 1 -- 172.21.15.45:0/1003345561 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5de409c8c0 con 0x7f5df4100e20 2024-09-10T10:14:46.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.578+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5df4101200 con 0x7f5df4100e20 2024-09-10T10:14:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.578+0000 7f5df1ffb640 1 -- 172.21.15.45:0/1003345561 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11843 (secure 0 0 0) 0x7f5de409c1c0 con 0x7f5df4100e20 2024-09-10T10:14:46.580 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:14:46.580 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":21,"btime":"2024-09-10T10:08:10:809804+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2062348864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2062348864},{"type":"v1","addr":"172.21.15.45:6835","nonce":2062348864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1314634033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1314634033},{"type":"v1","addr":"172.21.15.45:6837","nonce":1314634033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/743802171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":743802171},{"type":"v1","addr":"172.21.15.45:6839","nonce":743802171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14598,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/511130082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":511130082},{"type":"v1","addr":"172.21.15.45:6841","nonce":511130082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6833/531637764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6832","nonce":531637764},{"type":"v1","addr":"172.21.15.162:6833","nonce":531637764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2229149639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2229149639},{"type":"v1","addr":"172.21.15.47:6833","nonce":2229149639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/2176252394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":2176252394},{"type":"v1","addr":"172.21.15.47:6835","nonce":2176252394}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6835/1763949384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6834","nonce":1763949384},{"type":"v1","addr":"172.21.15.162:6835","nonce":1763949384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.162:6837/3408858695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6836","nonce":3408858695},{"type":"v1","addr":"172.21.15.162:6837","nonce":3408858695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2773219506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2773219506},{"type":"v1","addr":"172.21.15.47:6837","nonce":2773219506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T10:01:34.924889+0000","modified":"2024-09-10T10:08:10.809787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6839/2511396298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":2511396298},{"type":"v1","addr":"172.21.15.47:6839","nonce":2511396298}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25127":{"gid":25127,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.162:6839/2699160150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6838","nonce":2699160150},{"type":"v1","addr":"172.21.15.162:6839","nonce":2699160150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-10T10:14:46.580 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 21 2024-09-10T10:14:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5ddc0a2cf0 msgr2=0x7f5ddc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5ddc0a2cf0 0x7f5ddc0a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5df406aac0 tx=0x7f5de8005a20 comp rx=0 tx=0).stop 2024-09-10T10:14:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4100e20 msgr2=0x7f5df4073b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4100e20 0x7f5df4073b80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5de400e9f0 tx=0x7f5de400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 shutdown_connections 2024-09-10T10:14:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5ddc0a2cf0 0x7f5ddc0a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5df4102ef0 0x7f5df406f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5df4102570 0x7f5df406eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 --2- 172.21.15.45:0/1003345561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5df4100e20 0x7f5df4073b80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 >> 172.21.15.45:0/1003345561 conn(0x7f5df40fc7d0 msgr2=0x7f5df40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 shutdown_connections 2024-09-10T10:14:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:46.581+0000 7f5dfb0cf640 1 -- 172.21.15.45:0/1003345561 wait complete. 2024-09-10T10:14:47.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:14:47.529 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:47.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:47 smithi162 ceph-mon[34963]: pgmap v361: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:14:47.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:47 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1003345561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:14:47.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:47 smithi045 ceph-mon[31114]: pgmap v361: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:14:47.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:47 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1003345561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:14:47.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:47 smithi047 ceph-mon[35244]: pgmap v361: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:14:47.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:47 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1003345561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/3180539816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 msgr2=0x7f5c2410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/3180539816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c2410e610 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c009ca0 tx=0x7f5c1c02a950 comp rx=0 tx=0).stop 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/3180539816 shutdown_connections 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/3180539816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c2410e610 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/3180539816 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c24103660 0x7f5c2410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/3180539816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c24102d40 0x7f5c24103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/3180539816 >> 172.21.15.45:0/3180539816 conn(0x7f5c240fc7a0 msgr2=0x7f5c240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/3180539816 shutdown_connections 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.053+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/3180539816 wait complete. 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 Processor -- start 2024-09-10T10:14:48.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 -- start start 2024-09-10T10:14:48.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c24102d40 0x7f5c241009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:48.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c24103660 0x7f5c24100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:48.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c24101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24112000 con 0x7f5c2410c170 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24111e80 con 0x7f5c24102d40 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2bfc7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f5c24112180 con 0x7f5c24103660 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2a564640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c24101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c29d63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c24102d40 0x7f5c241009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c29d63640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c24102d40 0x7f5c241009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.45:33006/0 (socket says 172.21.15.45:33006) 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c29d63640 1 -- 172.21.15.45:0/2911016805 learned_addr learned my addr 172.21.15.45:0/2911016805 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c2a564640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c24101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55892/0 (socket says 172.21.15.45:55892) 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.054+0000 7f5c29562640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c24103660 0x7f5c24100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2a564640 1 -- 172.21.15.45:0/2911016805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c24103660 msgr2=0x7f5c24100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2a564640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c24103660 0x7f5c24100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2a564640 1 -- 172.21.15.45:0/2911016805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c24102d40 msgr2=0x7f5c241009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2a564640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c24102d40 0x7f5c241009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2a564640 1 -- 172.21.15.45:0/2911016805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c241a68f0 con 0x7f5c2410c170 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c29562640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c24103660 0x7f5c24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2a564640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c24101420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005e30 tx=0x7f5c1c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c1affd640 1 -- 172.21.15.45:0/2911016805 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c034040 con 0x7f5c2410c170 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c1affd640 1 -- 172.21.15.45:0/2911016805 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c1c02fe20 con 0x7f5c2410c170 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c241a6b80 con 0x7f5c2410c170 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c1affd640 1 -- 172.21.15.45:0/2911016805 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c004760 con 0x7f5c2410c170 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.055+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c241a7060 con 0x7f5c2410c170 2024-09-10T10:14:48.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.056+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c24103120 con 0x7f5c2410c170 2024-09-10T10:14:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.057+0000 7f5c1affd640 1 -- 172.21.15.45:0/2911016805 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5c1c037020 con 0x7f5c2410c170 2024-09-10T10:14:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.058+0000 7f5c1affd640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5c100a2cf0 0x7f5c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.058+0000 7f5c1affd640 1 -- 172.21.15.45:0/2911016805 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f5c1c0f0280 con 0x7f5c2410c170 2024-09-10T10:14:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.058+0000 7f5c29d63640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5c100a2cf0 0x7f5c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:48.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.059+0000 7f5c29d63640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5c100a2cf0 0x7f5c100a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5c20005fd0 tx=0x7f5c2000b480 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:48.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.061+0000 7f5c1affd640 1 -- 172.21.15.45:0/2911016805 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c1c0bcf90 con 0x7f5c2410c170 2024-09-10T10:14:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.236+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c241a7350 con 0x7f5c2410c170 2024-09-10T10:14:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.238+0000 7f5c1affd640 1 -- 172.21.15.45:0/2911016805 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23198 (secure 0 0 0) 0x7f5c1c0bc890 con 0x7f5c2410c170 2024-09-10T10:14:48.240 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-09-10T10:14:48.240 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"729b10fe-6f5a-11ef-bcea-c7b262605968","created":"2024-09-10T09:53:53.416864+0000","modified":"2024-09-10T10:14:16.271892+0000","last_up_change":"2024-09-10T10:00:46.790146+0000","last_in_change":"2024-09-10T10:00:22.204369+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:57:03.045230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:01:30.706328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:01:33.219365+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"edb5ca81-e36d-4acd-9f67-35e5fb644772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6803","nonce":378340133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6805","nonce":378340133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6809","nonce":378340133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":378340133},{"type":"v1","addr":"172.21.15.45:6807","nonce":378340133}]},"public_addr":"172.21.15.45:6803/378340133","cluster_addr":"172.21.15.45:6805/378340133","heartbeat_back_addr":"172.21.15.45:6809/378340133","heartbeat_front_addr":"172.21.15.45:6807/378340133","state":["exists","up"]},{"osd":1,"uuid":"20cd1eda-0149-4eab-a5ff-9463c2b556fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6801","nonce":3842809154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6803","nonce":3842809154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6807","nonce":3842809154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":3842809154},{"type":"v1","addr":"172.21.15.47:6805","nonce":3842809154}]},"public_addr":"172.21.15.47:6801/3842809154","cluster_addr":"172.21.15.47:6803/3842809154","heartbeat_back_addr":"172.21.15.47:6807/3842809154","heartbeat_front_addr":"172.21.15.47:6805/3842809154","state":["exists","up"]},{"osd":2,"uuid":"aaa52e4d-31b3-44e4-a0bb-91c2f7f497e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6801","nonce":572103448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6803","nonce":572103448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6807","nonce":572103448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":572103448},{"type":"v1","addr":"172.21.15.162:6805","nonce":572103448}]},"public_addr":"172.21.15.162:6801/572103448","cluster_addr":"172.21.15.162:6803/572103448","heartbeat_back_addr":"172.21.15.162:6807/572103448","heartbeat_front_addr":"172.21.15.162:6805/572103448","state":["exists","up"]},{"osd":3,"uuid":"14f9d3be-6d90-4490-b946-4a96c46736f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6811","nonce":2620350256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6813","nonce":2620350256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6817","nonce":2620350256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2620350256},{"type":"v1","addr":"172.21.15.45:6815","nonce":2620350256}]},"public_addr":"172.21.15.45:6811/2620350256","cluster_addr":"172.21.15.45:6813/2620350256","heartbeat_back_addr":"172.21.15.45:6817/2620350256","heartbeat_front_addr":"172.21.15.45:6815/2620350256","state":["exists","up"]},{"osd":4,"uuid":"3e9ec487-151e-4982-b522-3a2c3f90e407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6809","nonce":2470611139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6811","nonce":2470611139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6815","nonce":2470611139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":2470611139},{"type":"v1","addr":"172.21.15.47:6813","nonce":2470611139}]},"public_addr":"172.21.15.47:6809/2470611139","cluster_addr":"172.21.15.47:6811/2470611139","heartbeat_back_addr":"172.21.15.47:6815/2470611139","heartbeat_front_addr":"172.21.15.47:6813/2470611139","state":["exists","up"]},{"osd":5,"uuid":"086c844a-fd5a-4ded-9156-a2ccd034521f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6809","nonce":2110302929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6811","nonce":2110302929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6815","nonce":2110302929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":2110302929},{"type":"v1","addr":"172.21.15.162:6813","nonce":2110302929}]},"public_addr":"172.21.15.162:6809/2110302929","cluster_addr":"172.21.15.162:6811/2110302929","heartbeat_back_addr":"172.21.15.162:6815/2110302929","heartbeat_front_addr":"172.21.15.162:6813/2110302929","state":["exists","up"]},{"osd":6,"uuid":"29470075-95e2-48d8-a4f3-88b0b390d2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6819","nonce":135941807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6821","nonce":135941807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6825","nonce":135941807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":135941807},{"type":"v1","addr":"172.21.15.45:6823","nonce":135941807}]},"public_addr":"172.21.15.45:6819/135941807","cluster_addr":"172.21.15.45:6821/135941807","heartbeat_back_addr":"172.21.15.45:6825/135941807","heartbeat_front_addr":"172.21.15.45:6823/135941807","state":["exists","up"]},{"osd":7,"uuid":"32004f02-5b68-43bf-9af9-f9cabebe8043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6817","nonce":2101068416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6819","nonce":2101068416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6823","nonce":2101068416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":2101068416},{"type":"v1","addr":"172.21.15.47:6821","nonce":2101068416}]},"public_addr":"172.21.15.47:6817/2101068416","cluster_addr":"172.21.15.47:6819/2101068416","heartbeat_back_addr":"172.21.15.47:6823/2101068416","heartbeat_front_addr":"172.21.15.47:6821/2101068416","state":["exists","up"]},{"osd":8,"uuid":"3e8cc6aa-174a-4685-a735-e8a9aed57f04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6817","nonce":3586576942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6819","nonce":3586576942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6823","nonce":3586576942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":3586576942},{"type":"v1","addr":"172.21.15.162:6821","nonce":3586576942}]},"public_addr":"172.21.15.162:6817/3586576942","cluster_addr":"172.21.15.162:6819/3586576942","heartbeat_back_addr":"172.21.15.162:6823/3586576942","heartbeat_front_addr":"172.21.15.162:6821/3586576942","state":["exists","up"]},{"osd":9,"uuid":"12ab7cd4-9ec8-46b4-9a89-5d954e80b692","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6827","nonce":4106016815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6829","nonce":4106016815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6833","nonce":4106016815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":4106016815},{"type":"v1","addr":"172.21.15.45:6831","nonce":4106016815}]},"public_addr":"172.21.15.45:6827/4106016815","cluster_addr":"172.21.15.45:6829/4106016815","heartbeat_back_addr":"172.21.15.45:6833/4106016815","heartbeat_front_addr":"172.21.15.45:6831/4106016815","state":["exists","up"]},{"osd":10,"uuid":"eb823514-17df-4818-bfdf-7b8cdbcfbcd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6825","nonce":3408198504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6827","nonce":3408198504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6831","nonce":3408198504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":3408198504},{"type":"v1","addr":"172.21.15.47:6829","nonce":3408198504}]},"public_addr":"172.21.15.47:6825/3408198504","cluster_addr":"172.21.15.47:6827/3408198504","heartbeat_back_addr":"172.21.15.47:6831/3408198504","heartbeat_front_addr":"172.21.15.47:6829/3408198504","state":["exists","up"]},{"osd":11,"uuid":"147281ba-9297-46a4-b3a1-c12eaa3456b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6825","nonce":1692831932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6827","nonce":1692831932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6831","nonce":1692831932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":1692831932},{"type":"v1","addr":"172.21.15.162:6829","nonce":1692831932}]},"public_addr":"172.21.15.162:6825/1692831932","cluster_addr":"172.21.15.162:6827/1692831932","heartbeat_back_addr":"172.21.15.162:6831/1692831932","heartbeat_front_addr":"172.21.15.162:6829/1692831932","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:10.802745+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:56:35.778609+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:00.894768+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:25.987963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:57:51.475061+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:16.227903+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:58:41.126903+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:05.976728+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:30.502184+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:59:54.873980+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:19.372962+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:00:44.274032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/952575454":"2024-09-11T10:03:11.647760+0000","172.21.15.45:6800/952575454":"2024-09-11T10:03:11.647760+0000","172.21.15.45:0/3649460339":"2024-09-11T10:03:11.647760+0000","172.21.15.45:0/1519065487":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/3534138672":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/1473601885":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/444500118":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/384122965":"2024-09-11T10:03:11.647760+0000","172.21.15.45:6801/986599852":"2024-09-11T09:54:24.047197+0000","172.21.15.45:0/689699829":"2024-09-11T10:03:11.647760+0000","172.21.15.45:0/2521990938":"2024-09-11T09:54:48.533581+0000","172.21.15.45:6800/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/375260435":"2024-09-11T10:03:11.647760+0000","172.21.15.45:6801/1958743370":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/4168396686":"2024-09-11T09:54:48.533581+0000","172.21.15.45:0/21896051":"2024-09-11T10:03:11.647760+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5c100a2cf0 msgr2=0x7f5c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5c100a2cf0 0x7f5c100a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5c20005fd0 tx=0x7f5c2000b480 comp rx=0 tx=0).stop 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 msgr2=0x7f5c24101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c24101420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005e30 tx=0x7f5c1c003ce0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 shutdown_connections 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c2410c170 0x7f5c24101420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5c24103660 0x7f5c24100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f5c100a2cf0 0x7f5c100a51b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 --2- 172.21.15.45:0/2911016805 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5c24102d40 0x7f5c241009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 >> 172.21.15.45:0/2911016805 conn(0x7f5c240fc7a0 msgr2=0x7f5c240fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.240+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 shutdown_connections 2024-09-10T10:14:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:48.241+0000 7f5c2bfc7640 1 -- 172.21.15.45:0/2911016805 wait complete. 2024-09-10T10:14:48.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:48 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2911016805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:14:48.650 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:48 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2911016805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:14:48.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:48 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2911016805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:14:48.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T10:14:49.103 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:49.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:49 smithi045 ceph-mon[31114]: pgmap v362: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:14:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 -- 172.21.15.45:0/106431297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c101de0 msgr2=0x7f8e8c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 --2- 172.21.15.45:0/106431297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c101de0 0x7f8e8c1021c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009900 tx=0x7f8e8802a410 comp rx=0 tx=0).stop 2024-09-10T10:14:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 -- 172.21.15.45:0/106431297 shutdown_connections 2024-09-10T10:14:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 --2- 172.21.15.45:0/106431297 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e8c106f10 0x7f8e8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 --2- 172.21.15.45:0/106431297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8c102790 0x7f8e8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 --2- 172.21.15.45:0/106431297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c101de0 0x7f8e8c1021c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 -- 172.21.15.45:0/106431297 >> 172.21.15.45:0/106431297 conn(0x7f8e8c0fd8c0 msgr2=0x7f8e8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 -- 172.21.15.45:0/106431297 shutdown_connections 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.611+0000 7f8e9418e640 1 -- 172.21.15.45:0/106431297 wait complete. 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 Processor -- start 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 -- start start 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e8c101de0 0x7f8e8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8c102790 0x7f8e8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c106f10 0x7f8e8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c111f30 con 0x7f8e8c106f10 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c111db0 con 0x7f8e8c101de0 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9418e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8c1120b0 con 0x7f8e8c102790 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9272b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c106f10 0x7f8e8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9272b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c106f10 0x7f8e8c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55908/0 (socket says 172.21.15.45:55908) 2024-09-10T10:14:49.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e9272b640 1 -- 172.21.15.45:0/1515511331 learned_addr learned my addr 172.21.15.45:0/1515511331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e91f2a640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e8c101de0 0x7f8e8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.612+0000 7f8e91729640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8c102790 0x7f8e8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9272b640 1 -- 172.21.15.45:0/1515511331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8c102790 msgr2=0x7f8e8c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9272b640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8c102790 0x7f8e8c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9272b640 1 -- 172.21.15.45:0/1515511331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e8c101de0 msgr2=0x7f8e8c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9272b640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e8c101de0 0x7f8e8c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9272b640 1 -- 172.21.15.45:0/1515511331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8c1a67d0 con 0x7f8e8c106f10 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e91f2a640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e8c101de0 0x7f8e8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e91729640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8c102790 0x7f8e8c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9272b640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c106f10 0x7f8e8c074f70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400cce0 tx=0x7f8e8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e82ffd640 1 -- 172.21.15.45:0/1515511331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400f450 con 0x7f8e8c106f10 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8c1a6ac0 con 0x7f8e8c106f10 2024-09-10T10:14:49.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e82ffd640 1 -- 172.21.15.45:0/1515511331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e8400fad0 con 0x7f8e8c106f10 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e82ffd640 1 -- 172.21.15.45:0/1515511331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84018c10 con 0x7f8e8c106f10 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.613+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8c1a7000 con 0x7f8e8c106f10 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.614+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8c1021c0 con 0x7f8e8c106f10 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.615+0000 7f8e82ffd640 1 -- 172.21.15.45:0/1515511331 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8e8400f5f0 con 0x7f8e8c106f10 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.616+0000 7f8e82ffd640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8e780a2cf0 0x7f8e780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.616+0000 7f8e91f2a640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8e780a2cf0 0x7f8e780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.616+0000 7f8e82ffd640 1 -- 172.21.15.45:0/1515511331 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8556+0+0 (secure 0 0 0) 0x7f8e840cfbc0 con 0x7f8e8c106f10 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.616+0000 7f8e91f2a640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8e780a2cf0 0x7f8e780a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e88005ab0 tx=0x7f8e88005a20 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.620+0000 7f8e82ffd640 1 -- 172.21.15.45:0/1515511331 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e84014030 con 0x7f8e8c106f10 2024-09-10T10:14:49.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:49 smithi162 ceph-mon[34963]: pgmap v362: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:14:49.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:49 smithi047 ceph-mon[35244]: pgmap v362: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-09-10T10:14:49.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.832+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8e8c1a7230 con 0x7f8e8c106f10 2024-09-10T10:14:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.835+0000 7f8e82ffd640 1 -- 172.21.15.45:0/1515511331 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f8e8409c7c0 con 0x7f8e8c106f10 2024-09-10T10:14:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:14:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8e780a2cf0 msgr2=0x7f8e780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:49.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8e780a2cf0 0x7f8e780a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e88005ab0 tx=0x7f8e88005a20 comp rx=0 tx=0).stop 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c106f10 msgr2=0x7f8e8c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c106f10 0x7f8e8c074f70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400cce0 tx=0x7f8e8400a620 comp rx=0 tx=0).stop 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 shutdown_connections 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e8c106f10 0x7f8e8c074f70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e8c102790 0x7f8e8c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f8e780a2cf0 0x7f8e780a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 --2- 172.21.15.45:0/1515511331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e8c101de0 0x7f8e8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 >> 172.21.15.45:0/1515511331 conn(0x7f8e8c0fd8c0 msgr2=0x7f8e8c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 shutdown_connections 2024-09-10T10:14:49.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:49.838+0000 7f8e9418e640 1 -- 172.21.15.45:0/1515511331 wait complete. 2024-09-10T10:14:50.150 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:14:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i[64742]: ignoring --setuser ceph since I am not root 2024-09-10T10:14:50.150 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:14:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i[64742]: ignoring --setgroup ceph since I am not root 2024-09-10T10:14:50.150 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:14:49 smithi162 ceph-mds[64746]: -- 172.21.15.162:0/3336343333 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0fbeafc20 con 0x55e0fbe8d000 2024-09-10T10:14:50.150 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:14:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i[64742]: starting mds.i at 2024-09-10T10:14:50.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:49 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a[31110]: 2024-09-10T10:14:49.833+0000 7fb840afd640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:14:50.160 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:14:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b[66141]: ignoring --setuser ceph since I am not root 2024-09-10T10:14:50.160 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:14:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b[66141]: ignoring --setgroup ceph since I am not root 2024-09-10T10:14:50.160 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:14:49 smithi047 ceph-mds[66145]: -- 172.21.15.47:0/2419658179 <== mon.2 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c58a29860 con 0x558c58a06800 2024-09-10T10:14:50.160 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:14:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b[66141]: starting mds.b at 2024-09-10T10:14:50.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:50 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1515511331' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:14:50.407 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:50 smithi045 ceph-mon[31114]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:14:50.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:50 smithi045 ceph-mon[31114]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:14:50.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:50 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/1515511331' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:14:50.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:50 smithi045 ceph-mon[31114]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T10:14:50.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:50 smithi045 ceph-mon[31114]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-10T10:14:50.436 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:14:50.649 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:50 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1515511331' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:14:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:50 smithi162 ceph-mon[34963]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:14:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:50 smithi162 ceph-mon[34963]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:14:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:50 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/1515511331' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:14:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:50 smithi162 ceph-mon[34963]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T10:14:50.650 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:50 smithi162 ceph-mon[34963]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-10T10:14:50.659 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:50 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1515511331' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:14:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:50 smithi047 ceph-mon[35244]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:14:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:50 smithi047 ceph-mon[35244]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:14:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:50 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/1515511331' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:14:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:50 smithi047 ceph-mon[35244]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T10:14:50.660 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:50 smithi047 ceph-mon[35244]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-10T10:14:50.697 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:51.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 -- 172.21.15.45:0/3128171721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0100e20 msgr2=0x7f99c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:51.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 --2- 172.21.15.45:0/3128171721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0100e20 0x7f99c0101200 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009900 tx=0x7f99b002a480 comp rx=0 tx=0).stop 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 -- 172.21.15.45:0/3128171721 shutdown_connections 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 --2- 172.21.15.45:0/3128171721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c0102ef0 0x7f99c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 --2- 172.21.15.45:0/3128171721 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99c0102570 0x7f99c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 --2- 172.21.15.45:0/3128171721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0100e20 0x7f99c0101200 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 -- 172.21.15.45:0/3128171721 >> 172.21.15.45:0/3128171721 conn(0x7f99c00fc7d0 msgr2=0x7f99c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 -- 172.21.15.45:0/3128171721 shutdown_connections 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 -- 172.21.15.45:0/3128171721 wait complete. 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.181+0000 7f99c4957640 1 Processor -- start 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99c4957640 1 -- start start 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99c4957640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99c0100e20 0x7f99c01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:51.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99c4957640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c0102570 0x7f99c0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99c4957640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99c4957640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0110f80 con 0x7f99c0102ef0 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99c4957640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0110e00 con 0x7f99c0102570 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99c4957640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0111100 con 0x7f99c0100e20 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55926/0 (socket says 172.21.15.45:55926) 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 -- 172.21.15.45:0/2277047191 learned_addr learned my addr 172.21.15.45:0/2277047191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99be59c640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99c0100e20 0x7f99c01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bdd9b640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c0102570 0x7f99c0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 -- 172.21.15.45:0/2277047191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99c0100e20 msgr2=0x7f99c01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99c0100e20 0x7f99c01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 -- 172.21.15.45:0/2277047191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c0102570 msgr2=0x7f99c0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:51.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c0102570 0x7f99c0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.182+0000 7f99bed9d640 1 -- 172.21.15.45:0/2277047191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c01a4990 con 0x7f99c0102ef0 2024-09-10T10:14:51.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.183+0000 7f99bed9d640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f99b800eaa0 tx=0x7f99b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:51.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.183+0000 7f99af7fe640 1 -- 172.21.15.45:0/2277047191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b800cdb0 con 0x7f99c0102ef0 2024-09-10T10:14:51.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.183+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c01a4b60 con 0x7f99c0102ef0 2024-09-10T10:14:51.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.183+0000 7f99af7fe640 1 -- 172.21.15.45:0/2277047191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b8005020 con 0x7f99c0102ef0 2024-09-10T10:14:51.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.183+0000 7f99af7fe640 1 -- 172.21.15.45:0/2277047191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b80116b0 con 0x7f99c0102ef0 2024-09-10T10:14:51.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.183+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c01a5070 con 0x7f99c0102ef0 2024-09-10T10:14:51.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.184+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c01021b0 con 0x7f99c0102ef0 2024-09-10T10:14:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.185+0000 7f99af7fe640 1 -- 172.21.15.45:0/2277047191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f99b8005560 con 0x7f99c0102ef0 2024-09-10T10:14:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.186+0000 7f99af7fe640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f99a80a2cf0 0x7f99a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.187+0000 7f99be59c640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f99a80a2cf0 0x7f99a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.187+0000 7f99af7fe640 1 -- 172.21.15.45:0/2277047191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8598+0+0 (secure 0 0 0) 0x7f99b80cf6a0 con 0x7f99c0102ef0 2024-09-10T10:14:51.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.187+0000 7f99be59c640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f99a80a2cf0 0x7f99a80a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009230 tx=0x7f99b00058d0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:51.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.191+0000 7f99af7fe640 1 -- 172.21.15.45:0/2277047191 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99b8016030 con 0x7f99c0102ef0 2024-09-10T10:14:51.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.404+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f99c0109090 con 0x7f99c0102ef0 2024-09-10T10:14:51.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.846+0000 7f99af7fe640 1 -- 172.21.15.45:0/2277047191 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f99b809c270 con 0x7f99c0102ef0 2024-09-10T10:14:51.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f99a80a2cf0 msgr2=0x7f99a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f99a80a2cf0 0x7f99a80a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009230 tx=0x7f99b00058d0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0102ef0 msgr2=0x7f99c01a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f99b800eaa0 tx=0x7f99b800ef60 comp rx=0 tx=0).stop 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 shutdown_connections 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0102ef0 0x7f99c01a4450 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f99a80a2cf0 0x7f99a80a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99c0102570 0x7f99c0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 --2- 172.21.15.45:0/2277047191 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f99c0100e20 0x7f99c01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 >> 172.21.15.45:0/2277047191 conn(0x7f99c00fc7d0 msgr2=0x7f99c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.848+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 shutdown_connections 2024-09-10T10:14:51.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:51.849+0000 7f99c4957640 1 -- 172.21.15.45:0/2277047191 wait complete. 2024-09-10T10:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:51 smithi045 ceph-mon[31114]: pgmap v364: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:51 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.162:6838/3330927744,v1:172.21.15.162:6839/3330927744] up:boot 2024-09-10T10:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:51 smithi045 ceph-mon[31114]: mds.? [v2:172.21.15.47:6838/835562612,v1:172.21.15.47:6839/835562612] up:boot 2024-09-10T10:14:52.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:51 smithi045 ceph-mon[31114]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-10T10:14:52.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:51 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:14:52.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:51 smithi045 ceph-mon[31114]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:14:52.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:51 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2277047191' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:14:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:51 smithi162 ceph-mon[34963]: pgmap v364: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:14:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:51 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.162:6838/3330927744,v1:172.21.15.162:6839/3330927744] up:boot 2024-09-10T10:14:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:51 smithi162 ceph-mon[34963]: mds.? [v2:172.21.15.47:6838/835562612,v1:172.21.15.47:6839/835562612] up:boot 2024-09-10T10:14:52.151 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:51 smithi162 ceph-mon[34963]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-10T10:14:52.152 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:51 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:14:52.152 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:51 smithi162 ceph-mon[34963]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:14:52.152 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:51 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2277047191' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:14:52.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:51 smithi047 ceph-mon[35244]: pgmap v364: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:14:52.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:51 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.162:6838/3330927744,v1:172.21.15.162:6839/3330927744] up:boot 2024-09-10T10:14:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:51 smithi047 ceph-mon[35244]: mds.? [v2:172.21.15.47:6838/835562612,v1:172.21.15.47:6839/835562612] up:boot 2024-09-10T10:14:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:51 smithi047 ceph-mon[35244]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-10T10:14:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:51 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:14:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:51 smithi047 ceph-mon[35244]: from='mgr.15060 172.21.15.45:0/3782965780' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:14:52.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:51 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2277047191' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:14:52.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:14:52.791 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:52 smithi162 ceph-mon[34963]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:14:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:52 smithi162 ceph-mon[34963]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:14:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:52 smithi162 ceph-mon[34963]: Cluster is now healthy 2024-09-10T10:14:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:52 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/2277047191' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:14:53.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:52 smithi162 ceph-mon[34963]: fsmap 12 up:standby 2024-09-10T10:14:53.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:52 smithi045 ceph-mon[31114]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:14:53.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:52 smithi045 ceph-mon[31114]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:14:53.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:52 smithi045 ceph-mon[31114]: Cluster is now healthy 2024-09-10T10:14:53.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:52 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/2277047191' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:14:53.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:52 smithi045 ceph-mon[31114]: fsmap 12 up:standby 2024-09-10T10:14:53.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:52 smithi047 ceph-mon[35244]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:14:53.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:52 smithi047 ceph-mon[35244]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:14:53.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:52 smithi047 ceph-mon[35244]: Cluster is now healthy 2024-09-10T10:14:53.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:52 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/2277047191' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:14:53.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:52 smithi047 ceph-mon[35244]: fsmap 12 up:standby 2024-09-10T10:14:53.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 -- 172.21.15.45:0/2326026483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8102d40 msgr2=0x7fb5d8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:53.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/2326026483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8102d40 0x7fb5d8103120 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009900 tx=0x7fb5d402a480 comp rx=0 tx=0).stop 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 -- 172.21.15.45:0/2326026483 shutdown_connections 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/2326026483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d810c170 0x7fb5d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/2326026483 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d8103660 0x7fb5d810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/2326026483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8102d40 0x7fb5d8103120 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 -- 172.21.15.45:0/2326026483 >> 172.21.15.45:0/2326026483 conn(0x7fb5d80fc7a0 msgr2=0x7fb5d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 -- 172.21.15.45:0/2326026483 shutdown_connections 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 -- 172.21.15.45:0/2326026483 wait complete. 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.255+0000 7fb5e00b0640 1 Processor -- start 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5e00b0640 1 -- start start 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5e00b0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d8102d40 0x7fb5d81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5e00b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8103660 0x7fb5d8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5e00b0640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d810c170 0x7fb5d819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:53.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5e00b0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8111f70 con 0x7fb5d8103660 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5e00b0640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8111df0 con 0x7fb5d810c170 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5e00b0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d81120f0 con 0x7fb5d8102d40 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8103660 0x7fb5d8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8103660 0x7fb5d8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48710/0 (socket says 172.21.15.45:48710) 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 -- 172.21.15.45:0/3070728794 learned_addr learned my addr 172.21.15.45:0/3070728794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dde4c640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d8102d40 0x7fb5d81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5de64d640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d810c170 0x7fb5d819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 -- 172.21.15.45:0/3070728794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d8102d40 msgr2=0x7fb5d81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d8102d40 0x7fb5d81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 -- 172.21.15.45:0/3070728794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d810c170 msgr2=0x7fb5d819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d810c170 0x7fb5d819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 -- 172.21.15.45:0/3070728794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d819d4a0 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5de64d640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d810c170 0x7fb5d819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.256+0000 7fb5dd64b640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8103660 0x7fb5d8198a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800d750 tx=0x7fb5c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.257+0000 7fb5ceffd640 1 -- 172.21.15.45:0/3070728794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8017070 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.257+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d819d790 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.257+0000 7fb5ceffd640 1 -- 172.21.15.45:0/3070728794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5c8005020 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.257+0000 7fb5ceffd640 1 -- 172.21.15.45:0/3070728794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8004d00 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.257+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d81a5070 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.258+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d8103120 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.259+0000 7fb5ceffd640 1 -- 172.21.15.45:0/3070728794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb5c8009a30 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.259+0000 7fb5ceffd640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c40a2cf0 0x7fb5c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.260+0000 7fb5dde4c640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c40a2cf0 0x7fb5c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.260+0000 7fb5ceffd640 1 -- 172.21.15.45:0/3070728794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8598+0+0 (secure 0 0 0) 0x7fb5c80cf370 con 0x7fb5d8103660 2024-09-10T10:14:53.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.260+0000 7fb5dde4c640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c40a2cf0 0x7fb5c40a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009230 tx=0x7fb5d40058d0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:53.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.263+0000 7fb5ceffd640 1 -- 172.21.15.45:0/3070728794 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5c809bee0 con 0x7fb5d8103660 2024-09-10T10:14:53.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.440+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb5d81071e0 con 0x7fb5d8103660 2024-09-10T10:14:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.853+0000 7fb5ceffd640 1 -- 172.21.15.45:0/3070728794 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7fb5c809b7e0 con 0x7fb5d8103660 2024-09-10T10:14:53.857 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c40a2cf0 msgr2=0x7fb5c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c40a2cf0 0x7fb5c40a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009230 tx=0x7fb5d40058d0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8103660 msgr2=0x7fb5d8198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8103660 0x7fb5d8198a30 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800d750 tx=0x7fb5c800dc10 comp rx=0 tx=0).stop 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 shutdown_connections 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7fb5c40a2cf0 0x7fb5c40a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5d810c170 0x7fb5d819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5d8103660 0x7fb5d8198a30 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.856+0000 7fb5e00b0640 1 --2- 172.21.15.45:0/3070728794 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb5d8102d40 0x7fb5d81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:53.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.857+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 >> 172.21.15.45:0/3070728794 conn(0x7fb5d80fc7a0 msgr2=0x7fb5d80fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:53.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.857+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 shutdown_connections 2024-09-10T10:14:53.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:53.857+0000 7fb5e00b0640 1 -- 172.21.15.45:0/3070728794 wait complete. 2024-09-10T10:14:54.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:53 smithi045 ceph-mon[31114]: pgmap v365: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:14:54.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:53 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3070728794' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:14:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:53 smithi162 ceph-mon[34963]: pgmap v365: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:14:54.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:53 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3070728794' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:14:54.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:53 smithi047 ceph-mon[35244]: pgmap v365: 97 pgs: 97 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T10:14:54.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:53 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3070728794' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:14:54.557 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:14:54.819 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:14:55.355 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:54 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/3070728794' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:14:55.355 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:54 smithi047 ceph-mon[35244]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:14:55.355 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:54 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/3070728794' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:14:55.355 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:54 smithi162 ceph-mon[34963]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:14:55.356 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:54 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/3070728794' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:14:55.356 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:54 smithi045 ceph-mon[31114]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:14:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.336+0000 7f238d714640 1 -- 172.21.15.45:0/3851189752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 msgr2=0x7f2388106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.336+0000 7f238d714640 1 --2- 172.21.15.45:0/3851189752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f2388106e60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2378009930 tx=0x7f237802a420 comp rx=0 tx=0).stop 2024-09-10T10:14:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 -- 172.21.15.45:0/3851189752 shutdown_connections 2024-09-10T10:14:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 --2- 172.21.15.45:0/3851189752 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2388101420 0x7f238810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 --2- 172.21.15.45:0/3851189752 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2388100a80 0x7f2388100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 --2- 172.21.15.45:0/3851189752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f2388106e60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 -- 172.21.15.45:0/3851189752 >> 172.21.15.45:0/3851189752 conn(0x7f23880fc7a0 msgr2=0x7f23880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 -- 172.21.15.45:0/3851189752 shutdown_connections 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 -- 172.21.15.45:0/3851189752 wait complete. 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 Processor -- start 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.337+0000 7f238d714640 1 -- start start 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f238d714640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2388100a80 0x7f2388198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f238d714640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2388101420 0x7f23881989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f238d714640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f238819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f238d714640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2388110ea0 con 0x7f2388106a80 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f238d714640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2388110d20 con 0x7f2388100a80 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f238d714640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 -- 0x7f2388111020 con 0x7f2388101420 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f238819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:55.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f238819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48740/0 (socket says 172.21.15.45:48740) 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 -- 172.21.15.45:0/225341599 learned_addr learned my addr 172.21.15.45:0/225341599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f2386ffd640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2388100a80 0x7f2388198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f237ffff640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2388101420 0x7f23881989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 -- 172.21.15.45:0/225341599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2388101420 msgr2=0x7f23881989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2388101420 0x7f23881989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 -- 172.21.15.45:0/225341599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2388100a80 msgr2=0x7f2388198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2388100a80 0x7f2388198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 -- 172.21.15.45:0/225341599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238819d430 con 0x7f2388106a80 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f23877fe640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f238819cd50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f238000cce0 tx=0x7f238000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.338+0000 7f2384ff9640 1 -- 172.21.15.45:0/225341599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238000f450 con 0x7f2388106a80 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.339+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238819d720 con 0x7f2388106a80 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.339+0000 7f2384ff9640 1 -- 172.21.15.45:0/225341599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f238000fad0 con 0x7f2388106a80 2024-09-10T10:14:55.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.339+0000 7f2384ff9640 1 -- 172.21.15.45:0/225341599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380018b60 con 0x7f2388106a80 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.339+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881a5000 con 0x7f2388106a80 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.339+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2388100ee0 con 0x7f2388106a80 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.341+0000 7f2384ff9640 1 -- 172.21.15.45:0/225341599 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f238000fc80 con 0x7f2388106a80 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.341+0000 7f2384ff9640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23740a2cf0 0x7f23740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.341+0000 7f2384ff9640 1 -- 172.21.15.45:0/225341599 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8102+0+0 (secure 0 0 0) 0x7f23800cfac0 con 0x7f2388106a80 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.341+0000 7f2386ffd640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23740a2cf0 0x7f23740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.342+0000 7f2386ffd640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23740a2cf0 0x7f23740a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2378009900 tx=0x7f23780058d0 comp rx=0 tx=0).ready entity=mgr.15060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:14:55.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.345+0000 7f2384ff9640 1 -- 172.21.15.45:0/225341599 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2380014030 con 0x7f2388106a80 2024-09-10T10:14:55.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.522+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f23881997e0 con 0x7f2388106a80 2024-09-10T10:14:55.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.859+0000 7f2384ff9640 1 -- 172.21.15.45:0/225341599 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7f23881997e0 con 0x7f2388106a80 2024-09-10T10:14:55.861 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-09-10T10:14:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.862+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23740a2cf0 msgr2=0x7f23740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.862+0000 7f238d714640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23740a2cf0 0x7f23740a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2378009900 tx=0x7f23780058d0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.862+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 msgr2=0x7f238819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:14:55.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.862+0000 7f238d714640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f238819cd50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f238000cce0 tx=0x7f238000a620 comp rx=0 tx=0).stop 2024-09-10T10:14:55.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.863+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 shutdown_connections 2024-09-10T10:14:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.863+0000 7f238d714640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2388106a80 0x7f238819cd50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.863+0000 7f238d714640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2388101420 0x7f23881989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.863+0000 7f238d714640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.45:6800/506901738,v1:172.21.15.45:6801/506901738] conn(0x7f23740a2cf0 0x7f23740a51b0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2378009900 tx=0x7f23780058d0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.863+0000 7f238d714640 1 --2- 172.21.15.45:0/225341599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2388100a80 0x7f2388198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:14:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.863+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 >> 172.21.15.45:0/225341599 conn(0x7f23880fc7a0 msgr2=0x7f238806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:14:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.864+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 shutdown_connections 2024-09-10T10:14:55.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-09-10T10:14:55.864+0000 7f238d714640 1 -- 172.21.15.45:0/225341599 wait complete. 2024-09-10T10:14:56.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:55 smithi162 ceph-mon[34963]: pgmap v367: 65 pgs: 65 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail 2024-09-10T10:14:56.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:55 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/225341599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:14:56.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:55 smithi045 ceph-mon[31114]: pgmap v367: 65 pgs: 65 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail 2024-09-10T10:14:56.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:55 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/225341599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:14:56.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:55 smithi047 ceph-mon[35244]: pgmap v367: 65 pgs: 65 active+clean; 21 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail 2024-09-10T10:14:56.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:55 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/225341599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:14:56.524 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:14:56.552 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:14:56.579 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:14:56.621 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T10:14:56.621 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:14:56.637 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:14:56.652 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:14:56.688 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T10:14:56.689 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T10:14:56.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a 2024-09-10T10:14:56.876 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:56 smithi045 systemd[1]: Stopping Ceph mon.a for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:14:57.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:56 smithi045 ceph-mon[31114]: from='client.? 172.21.15.45:0/225341599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:14:57.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:56 smithi045 ceph-mon[31114]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:14:57.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a[31110]: 2024-09-10T10:14:57.061+0000 7fb846308640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:14:57.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a[31110]: 2024-09-10T10:14:57.061+0000 7fb846308640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T10:14:57.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:56 smithi162 ceph-mon[34963]: from='client.? 172.21.15.45:0/225341599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:14:57.149 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:56 smithi162 ceph-mon[34963]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:14:57.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:56 smithi047 ceph-mon[35244]: from='client.? 172.21.15.45:0/225341599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:14:57.159 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:56 smithi047 ceph-mon[35244]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:14:57.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:57 smithi045 podman[90590]: 2024-09-10 10:14:57.135738659 +0000 UTC m=+0.191622760 container died ab3b8fe2420a38e21d48b516145674bf8fe66b23379607b0a03d295fd158fef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T10:14:57.774 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:57 smithi045 podman[90590]: 2024-09-10 10:14:57.572710457 +0000 UTC m=+0.628594540 container cleanup ab3b8fe2420a38e21d48b516145674bf8fe66b23379607b0a03d295fd158fef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T10:14:57.775 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:57 smithi045 bash[90590]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a 2024-09-10T10:14:58.112 INFO:journalctl@ceph.mon.a.smithi045.stdout:Sep 10 10:14:57 smithi045 podman[90602]: 2024-09-10 10:14:57.773995454 +0000 UTC m=+0.626847599 container remove ab3b8fe2420a38e21d48b516145674bf8fe66b23379607b0a03d295fd158fef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T10:14:58.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T10:14:58.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:14:58.183 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T10:14:58.183 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T10:14:58.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.b 2024-09-10T10:14:58.512 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:58 smithi047 systemd[1]: Stopping Ceph mon.b for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:14:58.838 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-b[35240]: 2024-09-10T10:14:58.511+0000 7f3398035640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:14:58.838 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-b[35240]: 2024-09-10T10:14:58.511+0000 7f3398035640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:14:58.838 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:58 smithi047 podman[71262]: 2024-09-10 10:14:58.541767782 +0000 UTC m=+0.128069260 container died 267f885128f0d6198c2d76ccc2e64402aa5a38064b1f1e67150d9e2bb228c081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:14:59.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:58 smithi047 podman[71262]: 2024-09-10 10:14:58.944371079 +0000 UTC m=+0.530672559 container cleanup 267f885128f0d6198c2d76ccc2e64402aa5a38064b1f1e67150d9e2bb228c081 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T10:14:59.160 INFO:journalctl@ceph.mon.b.smithi047.stdout:Sep 10 10:14:58 smithi047 bash[71262]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-b 2024-09-10T10:14:59.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T10:14:59.472 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:14:59.472 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T10:14:59.472 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T10:14:59.472 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.c 2024-09-10T10:14:59.778 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:59 smithi162 systemd[1]: Stopping Ceph mon.c for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:00.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-c[34959]: 2024-09-10T10:14:59.776+0000 7fe7756db640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:00.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-c[34959]: 2024-09-10T10:14:59.776+0000 7fe7756db640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:15:00.150 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:14:59 smithi162 podman[70336]: 2024-09-10 10:14:59.909118467 +0000 UTC m=+0.249111835 container died ca3816f4365a47fceb39b33020678df704a1218600f64f044f1a9913b5f68c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:00.554 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:15:00 smithi162 podman[70336]: 2024-09-10 10:15:00.326967469 +0000 UTC m=+0.666960836 container cleanup ca3816f4365a47fceb39b33020678df704a1218600f64f044f1a9913b5f68c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T10:15:00.554 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:15:00 smithi162 bash[70336]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-c 2024-09-10T10:15:00.805 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T10:15:00.827 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:15:00 smithi162 podman[70348]: 2024-09-10 10:15:00.55372353 +0000 UTC m=+0.645549589 container remove ca3816f4365a47fceb39b33020678df704a1218600f64f044f1a9913b5f68c29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:15:00.827 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:15:00 smithi162 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-10T10:15:00.827 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:15:00 smithi162 systemd[1]: Stopped Ceph mon.c for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:15:00.827 INFO:journalctl@ceph.mon.c.smithi162.stdout:Sep 10 10:15:00 smithi162 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mon.c.service: Consumed 12.439s CPU time. 2024-09-10T10:15:00.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:00.859 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T10:15:00.860 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T10:15:00.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.x 2024-09-10T10:15:01.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:15:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:15:00.674+0000 7fc61a78c640 -1 client.15072: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-10T10:15:01.150 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:15:00 smithi045 systemd[1]: Stopping Ceph mgr.x for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:01.513 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:15:01 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:15:01.169+0000 7fc6d706a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:01.513 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:15:01 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x[31338]: 2024-09-10T10:15:01.169+0000 7fc6d706a640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T10:15:01.513 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:15:01 smithi045 podman[90708]: 2024-09-10 10:15:01.210459764 +0000 UTC m=+0.148851925 container died 5586f3d8900fc5466389422529797bd34bcddd63e6ef7a5417043badb5372805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T10:15:01.863 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:15:01 smithi045 podman[90708]: 2024-09-10 10:15:01.619071093 +0000 UTC m=+0.557463245 container cleanup 5586f3d8900fc5466389422529797bd34bcddd63e6ef7a5417043badb5372805 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T10:15:01.863 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Sep 10 10:15:01 smithi045 bash[90708]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-x 2024-09-10T10:15:01.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T10:15:02.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:02.035 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T10:15:02.035 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T10:15:02.035 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.y 2024-09-10T10:15:02.410 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:15:02 smithi047 systemd[1]: Stopping Ceph mgr.y for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:02.742 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:15:02 smithi047 podman[71392]: 2024-09-10 10:15:02.430231308 +0000 UTC m=+0.196826397 container died dd075593474f4d01697da219a355f1762a3683b9e4b8a1b44e805b00388cc3ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:02.998 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:15:02 smithi047 podman[71392]: 2024-09-10 10:15:02.841174537 +0000 UTC m=+0.607769618 container cleanup dd075593474f4d01697da219a355f1762a3683b9e4b8a1b44e805b00388cc3ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:15:02.999 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:15:02 smithi047 bash[71392]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y 2024-09-10T10:15:03.301 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:15:03 smithi047 podman[71403]: 2024-09-10 10:15:03.067800866 +0000 UTC m=+0.641582462 container remove dd075593474f4d01697da219a355f1762a3683b9e4b8a1b44e805b00388cc3ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T10:15:03.301 INFO:journalctl@ceph.mgr.y.smithi047.stdout:Sep 10 10:15:03 smithi047 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T10:15:03.320 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T10:15:03.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:03.361 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T10:15:03.362 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T10:15:03.362 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.z 2024-09-10T10:15:03.649 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:15:03 smithi162 systemd[1]: Stopping Ceph mgr.z for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:03.991 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:15:03 smithi162 podman[70471]: 2024-09-10 10:15:03.712686741 +0000 UTC m=+0.158240142 container died 94a2f2b5237c25391e5e38ce914f9ac63863ce384a5b51e5add69700d68db61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T10:15:04.333 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:15:04 smithi162 podman[70471]: 2024-09-10 10:15:04.0647129 +0000 UTC m=+0.510266300 container cleanup 94a2f2b5237c25391e5e38ce914f9ac63863ce384a5b51e5add69700d68db61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T10:15:04.333 INFO:journalctl@ceph.mgr.z.smithi162.stdout:Sep 10 10:15:04 smithi162 bash[70471]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mgr-z 2024-09-10T10:15:04.502 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T10:15:04.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:04.538 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T10:15:04.538 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T10:15:04.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.0 2024-09-10T10:15:04.857 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:04 smithi045 systemd[1]: Stopping Ceph osd.0 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:05.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0[44916]: 2024-09-10T10:15:04.856+0000 7f1c9b805640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:05.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0[44916]: 2024-09-10T10:15:04.856+0000 7f1c9b805640 -1 osd.0 98 *** Got signal Terminated *** 2024-09-10T10:15:05.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0[44916]: 2024-09-10T10:15:04.856+0000 7f1c9b805640 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:15:10.874 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:10 smithi045 podman[90831]: 2024-09-10 10:15:10.538925192 +0000 UTC m=+5.804507840 container died 5e35b0a992672f81d975c06cb0de2ac5ebcb8c888ccbb1311a6f678e7de50910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:15:11.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:10 smithi045 podman[90831]: 2024-09-10 10:15:10.873362704 +0000 UTC m=+6.138945357 container cleanup 5e35b0a992672f81d975c06cb0de2ac5ebcb8c888ccbb1311a6f678e7de50910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T10:15:11.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:10 smithi045 bash[90831]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0 2024-09-10T10:15:11.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:11 smithi045 podman[90849]: 2024-09-10 10:15:11.033066932 +0000 UTC m=+0.489917289 container remove 5e35b0a992672f81d975c06cb0de2ac5ebcb8c888ccbb1311a6f678e7de50910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:15:11.737 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:11 smithi045 podman[90936]: 2024-09-10 10:15:11.364934802 +0000 UTC m=+0.016087605 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:15:11.737 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:11 smithi045 podman[90936]: 2024-09-10 10:15:11.500794048 +0000 UTC m=+0.151946895 container create 1c9120c3ab0cc46fbf9f4a90c7ba928f0ed6e257ef0296e8e2753ef26e48f91e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:15:11.737 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:11 smithi045 podman[90936]: 2024-09-10 10:15:11.736372304 +0000 UTC m=+0.387525110 container init 1c9120c3ab0cc46fbf9f4a90c7ba928f0ed6e257ef0296e8e2753ef26e48f91e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T10:15:12.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:11 smithi045 podman[90936]: 2024-09-10 10:15:11.739918027 +0000 UTC m=+0.391070832 container start 1c9120c3ab0cc46fbf9f4a90c7ba928f0ed6e257ef0296e8e2753ef26e48f91e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-10T10:15:12.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:11 smithi045 podman[90936]: 2024-09-10 10:15:11.81971735 +0000 UTC m=+0.470870169 container attach 1c9120c3ab0cc46fbf9f4a90c7ba928f0ed6e257ef0296e8e2753ef26e48f91e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:15:12.150 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:11 smithi045 podman[90936]: 2024-09-10 10:15:11.92900187 +0000 UTC m=+0.580154673 container died 1c9120c3ab0cc46fbf9f4a90c7ba928f0ed6e257ef0296e8e2753ef26e48f91e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T10:15:12.481 INFO:journalctl@ceph.osd.0.smithi045.stdout:Sep 10 10:15:12 smithi045 podman[90960]: 2024-09-10 10:15:12.456245856 +0000 UTC m=+0.520903120 container remove 1c9120c3ab0cc46fbf9f4a90c7ba928f0ed6e257ef0296e8e2753ef26e48f91e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T10:15:12.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T10:15:12.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:12.529 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T10:15:12.529 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T10:15:12.529 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.1 2024-09-10T10:15:12.847 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:12 smithi047 systemd[1]: Stopping Ceph osd.1 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:13.160 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1[39724]: 2024-09-10T10:15:12.845+0000 7f951cdec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:13.160 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1[39724]: 2024-09-10T10:15:12.845+0000 7f951cdec640 -1 osd.1 98 *** Got signal Terminated *** 2024-09-10T10:15:13.160 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1[39724]: 2024-09-10T10:15:12.845+0000 7f951cdec640 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:15:18.766 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:18 smithi047 podman[71517]: 2024-09-10 10:15:18.469292531 +0000 UTC m=+5.743380959 container died 02a376af6bacbae624cbe38c6051b053c5aef601e66509f8ea7f456ad7fddba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:15:19.115 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:18 smithi047 podman[71517]: 2024-09-10 10:15:18.846119064 +0000 UTC m=+6.120207484 container cleanup 02a376af6bacbae624cbe38c6051b053c5aef601e66509f8ea7f456ad7fddba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T10:15:19.115 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:18 smithi047 bash[71517]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1 2024-09-10T10:15:19.410 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:19 smithi047 podman[71533]: 2024-09-10 10:15:19.114419311 +0000 UTC m=+0.644588537 container remove 02a376af6bacbae624cbe38c6051b053c5aef601e66509f8ea7f456ad7fddba6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T10:15:19.776 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:19 smithi047 podman[71622]: 2024-09-10 10:15:19.448492229 +0000 UTC m=+0.099400684 container create 7a458e2a0c34e05cad54a77e29ce6a08f09d200bb42a8430d98324a458f6c328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:15:19.776 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:19 smithi047 podman[71622]: 2024-09-10 10:15:19.365053486 +0000 UTC m=+0.015961905 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:15:19.776 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:19 smithi047 podman[71622]: 2024-09-10 10:15:19.634149323 +0000 UTC m=+0.285057743 container init 7a458e2a0c34e05cad54a77e29ce6a08f09d200bb42a8430d98324a458f6c328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T10:15:19.776 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:19 smithi047 podman[71622]: 2024-09-10 10:15:19.637467515 +0000 UTC m=+0.288375929 container start 7a458e2a0c34e05cad54a77e29ce6a08f09d200bb42a8430d98324a458f6c328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-10T10:15:19.776 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:19 smithi047 podman[71622]: 2024-09-10 10:15:19.775641215 +0000 UTC m=+0.426549636 container attach 7a458e2a0c34e05cad54a77e29ce6a08f09d200bb42a8430d98324a458f6c328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:15:20.129 INFO:journalctl@ceph.osd.1.smithi047.stdout:Sep 10 10:15:19 smithi047 podman[71622]: 2024-09-10 10:15:19.834485055 +0000 UTC m=+0.485393465 container died 7a458e2a0c34e05cad54a77e29ce6a08f09d200bb42a8430d98324a458f6c328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:15:20.404 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T10:15:20.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:20.448 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T10:15:20.448 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T10:15:20.448 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.2 2024-09-10T10:15:20.877 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:20 smithi162 systemd[1]: Stopping Ceph osd.2 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:21.149 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2[39954]: 2024-09-10T10:15:20.875+0000 7f4487972640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:21.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2[39954]: 2024-09-10T10:15:20.875+0000 7f4487972640 -1 osd.2 98 *** Got signal Terminated *** 2024-09-10T10:15:21.150 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2[39954]: 2024-09-10T10:15:20.875+0000 7f4487972640 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:15:26.610 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:26 smithi162 podman[70598]: 2024-09-10 10:15:26.332865126 +0000 UTC m=+5.692923972 container died 9f3fc640bde998915a4cddaa0769a2a5242d661f562d120daeb74b227000b963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T10:15:26.900 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:26 smithi162 podman[70598]: 2024-09-10 10:15:26.708751025 +0000 UTC m=+6.068809877 container cleanup 9f3fc640bde998915a4cddaa0769a2a5242d661f562d120daeb74b227000b963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:15:26.900 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:26 smithi162 bash[70598]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2 2024-09-10T10:15:27.361 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:27 smithi162 podman[70616]: 2024-09-10 10:15:27.016827353 +0000 UTC m=+0.677131726 container remove 9f3fc640bde998915a4cddaa0769a2a5242d661f562d120daeb74b227000b963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:15:27.639 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:27 smithi162 podman[70705]: 2024-09-10 10:15:27.263540065 +0000 UTC m=+0.015753811 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:15:27.639 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:27 smithi162 podman[70705]: 2024-09-10 10:15:27.386010459 +0000 UTC m=+0.138224196 container create 30b183f79dd15ea3cc16ed4153ed3f24557ac5b7aef6a53ee1d82ff41e39db87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T10:15:27.900 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:27 smithi162 podman[70705]: 2024-09-10 10:15:27.638730956 +0000 UTC m=+0.390944706 container init 30b183f79dd15ea3cc16ed4153ed3f24557ac5b7aef6a53ee1d82ff41e39db87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:15:27.900 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:27 smithi162 podman[70705]: 2024-09-10 10:15:27.642678736 +0000 UTC m=+0.394892475 container start 30b183f79dd15ea3cc16ed4153ed3f24557ac5b7aef6a53ee1d82ff41e39db87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:27.900 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:27 smithi162 podman[70705]: 2024-09-10 10:15:27.72182281 +0000 UTC m=+0.474036564 container attach 30b183f79dd15ea3cc16ed4153ed3f24557ac5b7aef6a53ee1d82ff41e39db87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:15:27.900 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:27 smithi162 podman[70705]: 2024-09-10 10:15:27.833699553 +0000 UTC m=+0.585913292 container died 30b183f79dd15ea3cc16ed4153ed3f24557ac5b7aef6a53ee1d82ff41e39db87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:28.370 INFO:journalctl@ceph.osd.2.smithi162.stdout:Sep 10 10:15:28 smithi162 podman[70728]: 2024-09-10 10:15:28.350448839 +0000 UTC m=+0.510991161 container remove 30b183f79dd15ea3cc16ed4153ed3f24557ac5b7aef6a53ee1d82ff41e39db87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T10:15:28.376 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T10:15:28.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:28.418 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T10:15:28.418 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T10:15:28.418 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.3 2024-09-10T10:15:28.718 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:28 smithi045 systemd[1]: Stopping Ceph osd.3 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:28.989 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3[51844]: 2024-09-10T10:15:28.715+0000 7f443e724640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:28.989 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3[51844]: 2024-09-10T10:15:28.715+0000 7f443e724640 -1 osd.3 98 *** Got signal Terminated *** 2024-09-10T10:15:28.989 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:28 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3[51844]: 2024-09-10T10:15:28.715+0000 7f443e724640 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:15:31.409 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:31 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:31.155+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:32.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:32 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:32.167+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:33.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:32.863+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:33.650 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:33 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:33.391+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:33.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:33 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:33.198+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:33.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:33.481+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:34.399 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:33.911+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:34.535 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:34 smithi045 podman[91039]: 2024-09-10 10:15:34.260509029 +0000 UTC m=+5.641962464 container died b11061e943cfccc5ae8d6b208c8c7557e58f46f25a9986809d23afe020d30398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:15:34.536 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:34 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:34.406+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:34.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:34 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:34.234+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:34.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:34.513+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:34.900 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:34 smithi045 podman[91039]: 2024-09-10 10:15:34.633471576 +0000 UTC m=+6.014925014 container cleanup b11061e943cfccc5ae8d6b208c8c7557e58f46f25a9986809d23afe020d30398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:15:34.900 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:34 smithi045 bash[91039]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3 2024-09-10T10:15:35.266 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:34 smithi045 podman[91067]: 2024-09-10 10:15:34.943230361 +0000 UTC m=+0.677631210 container remove b11061e943cfccc5ae8d6b208c8c7557e58f46f25a9986809d23afe020d30398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:15:35.399 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:34.905+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:35.588 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:35 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:35.441+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:35.589 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:35 smithi045 podman[91154]: 2024-09-10 10:15:35.168537279 +0000 UTC m=+0.016497682 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:15:35.589 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:35 smithi045 podman[91154]: 2024-09-10 10:15:35.285786988 +0000 UTC m=+0.133747436 container create 9996f6678b7cd7645ea9d17f9d1f01fca10bef9f457033fe6b984a13271e94f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T10:15:35.589 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:35 smithi045 podman[91154]: 2024-09-10 10:15:35.50458287 +0000 UTC m=+0.352543273 container init 9996f6678b7cd7645ea9d17f9d1f01fca10bef9f457033fe6b984a13271e94f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T10:15:35.589 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:35 smithi045 podman[91154]: 2024-09-10 10:15:35.507914256 +0000 UTC m=+0.355874667 container start 9996f6678b7cd7645ea9d17f9d1f01fca10bef9f457033fe6b984a13271e94f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:35.589 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:35 smithi045 podman[91154]: 2024-09-10 10:15:35.587803205 +0000 UTC m=+0.435763623 container attach 9996f6678b7cd7645ea9d17f9d1f01fca10bef9f457033fe6b984a13271e94f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:15:35.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:35 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:35.192+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:35.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:35.556+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:35.900 INFO:journalctl@ceph.osd.3.smithi045.stdout:Sep 10 10:15:35 smithi045 podman[91154]: 2024-09-10 10:15:35.697725399 +0000 UTC m=+0.545685801 container died 9996f6678b7cd7645ea9d17f9d1f01fca10bef9f457033fe6b984a13271e94f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T10:15:36.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T10:15:36.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:36.333 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T10:15:36.333 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T10:15:36.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.4 2024-09-10T10:15:36.399 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:35.943+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:36.633 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:36 smithi047 systemd[1]: Stopping Ceph osd.4 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:36.633 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:36 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:36.200+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:36.650 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:36 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:36.425+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:36.650 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:36 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:36.463+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:36.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:36.506+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:36.909 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:36 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4[46695]: 2024-09-10T10:15:36.631+0000 7fadd25f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:36.910 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:36 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4[46695]: 2024-09-10T10:15:36.631+0000 7fadd25f9640 -1 osd.4 98 *** Got signal Terminated *** 2024-09-10T10:15:36.910 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:36 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4[46695]: 2024-09-10T10:15:36.631+0000 7fadd25f9640 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:15:37.399 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:36.977+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:37.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:37 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:37.191+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:37.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:37.551+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:37.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:37 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:37.459+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:37.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:37 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:37.417+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:38.399 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:37.969+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:38.650 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:38 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:38.478+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:38.650 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:38 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:38.368+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:38.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:38 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:38.186+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:38.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:38 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:38.566+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:39.399 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:38 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:38.964+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:39.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:39 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:39.189+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:39.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:39 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:39.535+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:39.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:39 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:39.520+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:39.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:39 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:39.402+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:40.399 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:39 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:39.938+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:40.409 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:40 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:40.145+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:40.896 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:40 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:40.500+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:40.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:40 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:40.567+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:40.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:40 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:40.423+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:40.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:40 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:40.423+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:41.149 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:40 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:40.895+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:41.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:40 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:40.895+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:41.385 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:41 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:41.112+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:41.385 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:41 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4[46695]: 2024-09-10T10:15:41.206+0000 7fadcebfe640 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.206648+0000 front 2024-09-10T10:15:15.206616+0000 (oldest deadline 2024-09-10T10:15:40.506814+0000) 2024-09-10T10:15:41.650 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:41 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:41.536+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:41.650 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:41 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:41.393+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:41.650 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:41 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:41.393+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:41.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:41 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:41.384+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:41.876 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:41 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:41.454+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:42.149 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:41 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:41.875+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:42.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:41 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:41.875+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:42.400 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:42 smithi047 podman[71741]: 2024-09-10 10:15:42.171614199 +0000 UTC m=+5.640893084 container died 823c044f6b9f90905eee3f4cfdd386c9c1f5a3d85661ce67f2b7322c629f89d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:15:42.401 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:42 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:42.134+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:42.401 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:42 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:42.134+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:42.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:42 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:42.399+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:42.660 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:42 smithi047 podman[71741]: 2024-09-10 10:15:42.548831727 +0000 UTC m=+6.018110616 container cleanup 823c044f6b9f90905eee3f4cfdd386c9c1f5a3d85661ce67f2b7322c629f89d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T10:15:42.660 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:42 smithi047 bash[71741]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4 2024-09-10T10:15:42.843 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:42 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:42.462+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:42.843 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:42 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:42.462+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:42.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:42 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:42.528+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:42.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:42 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:42.427+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:42.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:42 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:42.427+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:43.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:42 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:42.841+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:43.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:42 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:42.919+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:43.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:42 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:42.919+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:43.311 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:42 smithi047 podman[71759]: 2024-09-10 10:15:42.933404921 +0000 UTC m=+0.758745795 container remove 823c044f6b9f90905eee3f4cfdd386c9c1f5a3d85661ce67f2b7322c629f89d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T10:15:43.312 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:43 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:43.134+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:43.312 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:43 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:43.135+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:43.628 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:43 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:43.357+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:43.628 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:43 smithi047 podman[71846]: 2024-09-10 10:15:43.213742174 +0000 UTC m=+0.015815388 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:15:43.629 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:43 smithi047 podman[71846]: 2024-09-10 10:15:43.325657509 +0000 UTC m=+0.127730722 container create 0a9d2719dfa9d6683e82c62588b7df63d4f09962d21eaa670dae48917e74ec87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:15:43.629 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:43 smithi047 podman[71846]: 2024-09-10 10:15:43.544663829 +0000 UTC m=+0.346737043 container init 0a9d2719dfa9d6683e82c62588b7df63d4f09962d21eaa670dae48917e74ec87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T10:15:43.629 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:43 smithi047 podman[71846]: 2024-09-10 10:15:43.547985455 +0000 UTC m=+0.350058669 container start 0a9d2719dfa9d6683e82c62588b7df63d4f09962d21eaa670dae48917e74ec87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:15:43.629 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:43 smithi047 podman[71846]: 2024-09-10 10:15:43.628109558 +0000 UTC m=+0.430182784 container attach 0a9d2719dfa9d6683e82c62588b7df63d4f09962d21eaa670dae48917e74ec87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:43.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:43 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:43.422+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:43.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:43 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:43.422+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:43.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:43.575+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:43.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:43.469+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:43.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:43.469+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:43.909 INFO:journalctl@ceph.osd.4.smithi047.stdout:Sep 10 10:15:43 smithi047 podman[71846]: 2024-09-10 10:15:43.749984415 +0000 UTC m=+0.552057647 container died 0a9d2719dfa9d6683e82c62588b7df63d4f09962d21eaa670dae48917e74ec87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:15:44.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:43 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:43.796+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:44.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:43 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:43.910+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:44.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:43 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:43.910+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:44.323 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:44 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:44.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:44.323 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:44 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:44.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:44.343 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T10:15:44.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:44.383 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T10:15:44.384 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T10:15:44.384 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.5 2024-09-10T10:15:44.649 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:44 smithi162 systemd[1]: Stopping Ceph osd.5 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:44.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:44.377+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:44.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:44.377+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:44.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:44 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:44.332+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:44.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:44.506+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:44.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:44.506+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:44.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:44.614+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:44.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:44 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:44.614+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:45.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:44.826+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:45.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:44.701+0000 7fd858748640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:45.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:44.701+0000 7fd858748640 -1 osd.5 98 *** Got signal Terminated *** 2024-09-10T10:15:45.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:44.701+0000 7fd858748640 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:15:45.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:44.902+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:45.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:44 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:44.902+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:45.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:45 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:45.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:45.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:45 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:45.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:45.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:45 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:45.361+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:45.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:45 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:45.184+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:45.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:45 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:45.184+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:45.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:45.475+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:45.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:45.475+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:45.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:45.603+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:45.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:45 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:45.603+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:46.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:45 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:45.849+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:46.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:45 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:45.928+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:46.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:45 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:45.928+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:46.409 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:46 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:46.353+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:46.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:46 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:46.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:46.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:46 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:46.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:46.608 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:46.514+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:46.609 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:46.514+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:46.807 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:46.415+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:46.808 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:46.415+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:46.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:46.607+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:46.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:46.607+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:46.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:46 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:46.607+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:47.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:46.806+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:47.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:46.910+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:47.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:46 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:46.910+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:47.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:47 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:47.356+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:47.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:47 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:47.163+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:47.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:47 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:47.163+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:47.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:47 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:47.163+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:47.762 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:47.399+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:47.763 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:47.399+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:47.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:47.476+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:47.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:47.476+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:47.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:47.476+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:47.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:47.586+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:47.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:47.586+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:47.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:47 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:47.586+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:48.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:47.761+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:48.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:47.921+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:48.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:47.921+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:48.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:47 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:47.921+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.774293+0000 front 2024-09-10T10:15:23.774258+0000 (oldest deadline 2024-09-10T10:15:47.874436+0000) 2024-09-10T10:15:48.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:48.370+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:48.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:48.161+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:48.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:48.161+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:48.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:48 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:48.161+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:48.714 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:48.413+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:48.714 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:48.413+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:48.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:48.450+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:48.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:48.450+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:48.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:48.450+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:48.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:48.626+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:48.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:48.626+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:48.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:48.626+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:49.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:48.713+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:49.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:48.888+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.271677+0000 front 2024-09-10T10:15:09.271739+0000 (oldest deadline 2024-09-10T10:15:32.771583+0000) 2024-09-10T10:15:49.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:48.888+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.373233+0000 front 2024-09-10T10:15:17.373099+0000 (oldest deadline 2024-09-10T10:15:40.873213+0000) 2024-09-10T10:15:49.150 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:48 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5[46427]: 2024-09-10T10:15:48.888+0000 7fd850d45640 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.774293+0000 front 2024-09-10T10:15:23.774258+0000 (oldest deadline 2024-09-10T10:15:47.874436+0000) 2024-09-10T10:15:49.385 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:49.129+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:49.385 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:49.129+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:49.385 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:49.129+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:49.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:49.377+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:49.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:49.377+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:49.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:49.377+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:49.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:49 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:49.385+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:49.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:49 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:49.426+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:49.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:49 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:49.426+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:49.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:49 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:49.426+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:49.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:49 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:49.583+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:49.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:49 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:49.583+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:49.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:49 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:49.583+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:50.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:49 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:49.689+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:50.544 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:50 smithi162 podman[70820]: 2024-09-10 10:15:50.261988795 +0000 UTC m=+5.682581884 container died 4c258e15f6324bf5029abad50e73fe3f1ec7fa239b1c2d4ef57f218f43c085bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T10:15:50.544 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:50.393+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:50.544 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:50.393+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:50.544 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:50.393+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:50.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:50 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:50.359+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:50.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:50 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:50.166+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:50.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:50 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:50.166+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:50.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:50 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:50.166+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:50.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:50 smithi162 podman[70820]: 2024-09-10 10:15:50.651211413 +0000 UTC m=+6.071804498 container cleanup 4c258e15f6324bf5029abad50e73fe3f1ec7fa239b1c2d4ef57f218f43c085bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:50.900 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:50 smithi162 bash[70820]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5 2024-09-10T10:15:50.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:50.719+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:50.900 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:50 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:50.719+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:50.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:50.435+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:50.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:50.435+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:50.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:50.435+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:50.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:50.592+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:50.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:50.592+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:50.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:50 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:50.592+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:51.260 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:50 smithi162 podman[70837]: 2024-09-10 10:15:50.911131976 +0000 UTC m=+0.643968978 container remove 4c258e15f6324bf5029abad50e73fe3f1ec7fa239b1c2d4ef57f218f43c085bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:51.589 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:51 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:51.388+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:51.590 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:51 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:51.388+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:51.590 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:51 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:51.388+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:51.590 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:51 smithi162 podman[70925]: 2024-09-10 10:15:51.162169091 +0000 UTC m=+0.016076559 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:15:51.590 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:51 smithi162 podman[70925]: 2024-09-10 10:15:51.261979335 +0000 UTC m=+0.115886841 container create f2c87aec2d4c1f4f5f2d69756ddb0364db8fcaa57c453d872cc83f27ed6e2923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:15:51.590 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:51 smithi162 podman[70925]: 2024-09-10 10:15:51.489198768 +0000 UTC m=+0.343106228 container init f2c87aec2d4c1f4f5f2d69756ddb0364db8fcaa57c453d872cc83f27ed6e2923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:15:51.590 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:51 smithi162 podman[70925]: 2024-09-10 10:15:51.492963322 +0000 UTC m=+0.346870782 container start f2c87aec2d4c1f4f5f2d69756ddb0364db8fcaa57c453d872cc83f27ed6e2923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:15:51.590 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:51 smithi162 podman[70925]: 2024-09-10 10:15:51.589308629 +0000 UTC m=+0.443216101 container attach f2c87aec2d4c1f4f5f2d69756ddb0364db8fcaa57c453d872cc83f27ed6e2923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T10:15:51.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:51.345+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:51.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:51.199+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:51.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:51.199+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:51.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:51 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:51.199+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:51.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:51 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:51.440+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:51.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:51 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:51.440+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:51.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:51 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:51.440+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:51.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:51 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:51.546+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:51.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:51 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:51.546+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:51.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:51 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:51.546+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:51.902 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 10:15:51 smithi162 podman[70925]: 2024-09-10 10:15:51.679558976 +0000 UTC m=+0.533466434 container died f2c87aec2d4c1f4f5f2d69756ddb0364db8fcaa57c453d872cc83f27ed6e2923 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T10:15:51.902 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:51 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:51.761+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:51.902 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:51 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:51.761+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:52.269 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T10:15:52.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:15:52.308 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T10:15:52.309 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T10:15:52.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.6 2024-09-10T10:15:52.594 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:52.471+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:52.471+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:52.471+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:52 smithi045 systemd[1]: Stopping Ceph osd.6 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:52.551+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:52.551+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:52.551+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:52.593+0000 7f4c563de640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:52.593+0000 7f4c563de640 -1 osd.6 98 *** Got signal Terminated *** 2024-09-10T10:15:52.595 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:52.593+0000 7f4c563de640 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:15:52.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:52 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:52.374+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:52.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:52 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:52.374+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:52.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:52 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:52.374+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:52.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:52.362+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:52.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:52.362+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:52.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:52.174+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:52.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:52.174+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:52.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:52 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:52.174+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:53.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:52 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:52.782+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:53.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:52 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:52.782+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:53.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:53 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:53.411+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:53.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:53 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:53.411+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:53.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:53 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:53.200+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:53.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:53 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:53.200+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:53.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:53 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:53.200+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:53.784 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:53 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:53.411+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:53.784 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:53 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:53.411+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:53.784 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:53 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:53.411+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:53.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:53 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:53.511+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:53.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:53 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:53.511+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:53.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:53 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:53.511+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:53.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:53 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:53.509+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:53.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:53 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:53.509+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:53.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:53 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:53.509+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:54.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:53 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:53.782+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:54.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:53 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:53.782+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:54.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:54 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:54.363+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:54.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:54 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:54.363+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:54.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:54 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:54.363+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:54.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:54 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:54.455+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:54.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:54 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:54.455+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:54.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:54 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:54.211+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:54.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:54 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:54.211+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:54.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:54 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:54.211+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:54.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:54 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:54.486+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:54.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:54 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:54.486+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:54.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:54 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:54.486+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:54.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:54 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:54.466+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:54.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:54 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:54.466+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:54.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:54 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:54.466+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:55.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:54 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:54.777+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:55.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:54 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:54.777+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:55.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:55 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:55.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:55.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:55 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:55.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:55.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:55 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:55.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:55.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:55 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:55.413+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:55.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:55 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:55.413+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:55.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:55 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:55.172+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:55.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:55 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:55.172+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:55.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:55 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:55.172+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:55.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:55.502+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:55.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:55.502+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:55.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:55.502+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:55.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:55.434+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:55.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:55.434+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:55.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:55 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:55.434+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:56.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:55 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:55.755+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:56.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:55 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:55.755+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:56.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:56 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:56.341+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:56.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:56 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:56.341+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:56.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:56 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:56.341+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:56.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:56 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:56.425+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:56.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:56 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:56.425+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:56.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:56 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:56.217+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:56.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:56 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:56.217+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:56.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:56 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:56.217+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:56.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:56 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:56.477+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:56.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:56 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:56.477+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:56.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:56 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:56.477+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:56.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:56 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:56.447+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:56.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:56 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:56.447+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:56.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:56 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:56.447+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:57.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:56 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:56.771+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:57.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:56 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:56.771+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:57.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:56 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:56.771+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:15:57.475 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:57.222+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:57.475 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:57.222+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:57.476 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:57.222+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:57.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:57 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:57.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:57.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:57 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:57.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:57.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:57 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:57.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:57.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:57.433+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:07.896870+0000 front 2024-09-10T10:15:07.897064+0000 (oldest deadline 2024-09-10T10:15:33.097291+0000) 2024-09-10T10:15:57.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:57.433+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:17.798714+0000 front 2024-09-10T10:15:17.798592+0000 (oldest deadline 2024-09-10T10:15:43.698536+0000) 2024-09-10T10:15:57.900 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:57.433+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:25.399707+0000 front 2024-09-10T10:15:25.399674+0000 (oldest deadline 2024-09-10T10:15:46.499513+0000) 2024-09-10T10:15:57.901 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6[58793]: 2024-09-10T10:15:57.433+0000 7f4c529e3640 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.800403+0000 front 2024-09-10T10:15:31.800422+0000 (oldest deadline 2024-09-10T10:15:56.500491+0000) 2024-09-10T10:15:57.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:57.497+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:57.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:57.497+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:57.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:57.497+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:57.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:57 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:57.497+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:15:57.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:57.474+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:57.910 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:57.474+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:57.910 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:57.474+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:15:58.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:57 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:57.772+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:58.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:57 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:57.772+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:58.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:57 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:57.772+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:15:58.400 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:58 smithi045 podman[91255]: 2024-09-10 10:15:58.043020535 +0000 UTC m=+5.538468237 container died 8bcbcd543097a160258631235bbbd6ee385864a7bad56a97482d870aace862a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T10:15:58.450 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:58.176+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:58.450 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:58.176+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:58.450 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:58.176+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:58.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:58 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:58.324+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:58.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:58 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:58.324+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:58.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:58 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:58.324+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:58.795 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:58 smithi045 podman[91255]: 2024-09-10 10:15:58.543217192 +0000 UTC m=+6.038664935 container cleanup 8bcbcd543097a160258631235bbbd6ee385864a7bad56a97482d870aace862a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T10:15:58.795 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:58 smithi045 bash[91255]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6 2024-09-10T10:15:58.795 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:58 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:58.545+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:58.795 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:58 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:58.545+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:58.795 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:58 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:58.545+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:58.795 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:58 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:58.545+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:15:58.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:58.448+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:58.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:58.448+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:58.910 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:58 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:58.448+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:15:59.109 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:58 smithi045 podman[91271]: 2024-09-10 10:15:58.794401952 +0000 UTC m=+0.754788683 container remove 8bcbcd543097a160258631235bbbd6ee385864a7bad56a97482d870aace862a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:15:59.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:58 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:58.756+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:15:59.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:58 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:58.756+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:15:59.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:58 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:58.756+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:15:59.400 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:59 smithi045 podman[91362]: 2024-09-10 10:15:59.011257758 +0000 UTC m=+0.016419499 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:15:59.400 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:59 smithi045 podman[91362]: 2024-09-10 10:15:59.109789119 +0000 UTC m=+0.114950847 container create 227a1d671561781c415a0721a13419a7a5cf50c703ced309487f6379a7ba21aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T10:15:59.400 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:59 smithi045 podman[91362]: 2024-09-10 10:15:59.34143176 +0000 UTC m=+0.346593488 container init 227a1d671561781c415a0721a13419a7a5cf50c703ced309487f6379a7ba21aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T10:15:59.400 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:59 smithi045 podman[91362]: 2024-09-10 10:15:59.344826152 +0000 UTC m=+0.349987884 container start 227a1d671561781c415a0721a13419a7a5cf50c703ced309487f6379a7ba21aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T10:15:59.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:59 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:59.206+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:15:59.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:59 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:59.206+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:15:59.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:59 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:59.206+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:15:59.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:15:59 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:15:59.206+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:15:59.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:59.298+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:15:59.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:59.298+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:15:59.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:15:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:15:59.298+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:15:59.848 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:59 smithi045 podman[91362]: 2024-09-10 10:15:59.427496654 +0000 UTC m=+0.432658394 container attach 227a1d671561781c415a0721a13419a7a5cf50c703ced309487f6379a7ba21aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T10:15:59.848 INFO:journalctl@ceph.osd.6.smithi045.stdout:Sep 10 10:15:59 smithi045 podman[91362]: 2024-09-10 10:15:59.540079807 +0000 UTC m=+0.545241540 container died 227a1d671561781c415a0721a13419a7a5cf50c703ced309487f6379a7ba21aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T10:15:59.848 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:59 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:59.558+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:15:59.848 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:59 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:59.558+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:15:59.848 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:59 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:59.558+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:15:59.848 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:15:59 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:15:59.558+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:15:59.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:59 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:59.483+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:15:59.910 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:59 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:59.483+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:15:59.910 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:15:59 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:15:59.483+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:16:00.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:59.750+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:00.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:59.750+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:00.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:15:59 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:15:59.750+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:00.169 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T10:16:00.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:00.212 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T10:16:00.212 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T10:16:00.212 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.7 2024-09-10T10:16:00.447 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:00.164+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:00.447 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:00.164+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:00.447 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:00.164+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:00.448 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:00.164+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:00.448 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:00 smithi047 systemd[1]: Stopping Ceph osd.7 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:00.448 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:00.445+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:16:00.448 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:00.445+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:16:00.448 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:00.445+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:16:00.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:00 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:00.289+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:00.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:00 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:00.289+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:00.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:00 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:00.289+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:00.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:00.603+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:00.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:00.603+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:00.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:00.603+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:00.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:00 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:00.603+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:00.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:00.541+0000 7f33cbd8e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:00.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:00.541+0000 7f33cbd8e640 -1 osd.7 98 *** Got signal Terminated *** 2024-09-10T10:16:00.910 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:00 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:00.541+0000 7f33cbd8e640 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:16:01.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:00 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:00.747+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:01.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:00 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:00.747+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:01.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:00 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:00.747+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:01.496 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:01 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:01.185+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:01.496 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:01 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:01.185+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:01.496 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:01 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:01.185+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:01.497 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:01 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:01.185+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:01.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:01.331+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:01.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:01.331+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:01.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:01.331+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:01.783 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:01 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:01.495+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:16:01.783 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:01 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:01.495+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:16:01.783 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:01 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:01.495+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:16:01.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:01 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:01.635+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:01.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:01 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:01.635+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:01.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:01 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:01.635+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:01.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:01 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:01.635+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:02.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:01.781+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:02.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:01.781+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:02.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:01 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:01.781+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:02.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:02 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:02.162+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:02.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:02 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:02.162+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:02.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:02 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:02.162+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:02.485 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:02 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:02.162+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:02.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:02 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:02.295+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:02.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:02 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:02.295+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:02.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:02 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:02.295+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:02.772 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:02 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:02.483+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:16:02.772 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:02 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:02.483+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:16:02.772 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:02 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:02.483+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:16:02.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:02 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:02.638+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:02.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:02 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:02.638+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:02.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:02 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:02.638+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:02.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:02 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:02.638+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:03.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:02 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:02.821+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:03.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:02 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:02.821+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:03.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:02 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:02.821+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:03.424 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:03 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:03.320+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:03.424 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:03 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:03.320+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:03.424 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:03 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:03.320+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:03.473 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:03.208+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:03.474 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:03.208+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:03.474 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:03.208+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:03.474 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:03.208+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:03.474 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:03.208+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:03.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:03 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:03.590+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:03.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:03 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:03.590+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:03.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:03 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:03.590+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:03.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:03 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:03.590+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:03.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:03.472+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:16:03.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:03.472+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:16:03.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:03.472+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:16:04.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:03 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:03.808+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:04.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:03 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:03.808+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:04.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:03 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:03.808+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:04.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:03 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:03.808+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:04.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:04 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:04.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:04.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:04 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:04.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:04.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:04 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:04.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:04.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:04.478+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:16:04.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:04.478+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:16:04.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:04.478+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:16:04.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:04.242+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:04.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:04.242+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:04.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:04.242+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:04.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:04.242+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:04.661 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:04 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:04.242+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:04.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:04.597+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:04.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:04.597+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:04.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:04.597+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:04.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:04 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:04.597+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:05.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:04 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:04.796+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:05.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:04 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:04.796+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:05.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:04 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:04.796+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:05.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:04 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:04.796+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:05.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:05 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:05.325+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:05.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:05 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:05.325+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:05.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:05 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:05.325+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:05.659 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:05.456+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.550510+0000 front 2024-09-10T10:15:15.550691+0000 (oldest deadline 2024-09-10T10:15:40.850648+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:05.456+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:20.851124+0000 front 2024-09-10T10:15:20.851228+0000 (oldest deadline 2024-09-10T10:15:51.451755+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:05.456+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.452345+0000 front 2024-09-10T10:15:31.452355+0000 (oldest deadline 2024-09-10T10:15:57.352221+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7[53696]: 2024-09-10T10:16:05.456+0000 7f33c438b640 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.654330+0000 front 2024-09-10T10:15:40.654377+0000 (oldest deadline 2024-09-10T10:16:04.754659+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:05.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:05.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:05.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:05.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:05.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:05 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:05.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:05.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:05 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:05.607+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:05.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:05 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:05.607+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:05.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:05 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:05.607+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:05.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:05 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:05.607+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:06.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:05 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:05.845+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:06.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:05 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:05.845+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:06.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:05 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:05.845+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:06.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:05 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:05.845+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:06.401 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:06 smithi047 podman[71950]: 2024-09-10 10:16:06.140242968 +0000 UTC m=+5.726120739 container died 152744f346c415c03c801e38cfbc9db531fc81f709ca236ca14b950d4fdb63c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T10:16:06.402 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:06.329+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:06.402 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:06.329+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:06.402 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:06.329+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:06.402 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:06.329+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:06.402 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:06 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:06.329+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:06.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:06.354+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:06.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:06.354+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:06.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:06.354+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:06.652 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:06 smithi047 podman[71950]: 2024-09-10 10:16:06.484096852 +0000 UTC m=+6.069974612 container cleanup 152744f346c415c03c801e38cfbc9db531fc81f709ca236ca14b950d4fdb63c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T10:16:06.652 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:06 smithi047 bash[71950]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7 2024-09-10T10:16:06.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:06.627+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:06.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:06.627+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:06.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:06.627+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:06.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:06 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:06.627+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:06.909 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:06 smithi047 podman[71977]: 2024-09-10 10:16:06.651862473 +0000 UTC m=+0.509691144 container remove 152744f346c415c03c801e38cfbc9db531fc81f709ca236ca14b950d4fdb63c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:16:07.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:06.830+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:07.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:06.830+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:07.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:06.830+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:07.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:06 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:06.830+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:07.342 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:07 smithi047 podman[72067]: 2024-09-10 10:16:06.970156634 +0000 UTC m=+0.016046780 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:16:07.342 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:07 smithi047 podman[72067]: 2024-09-10 10:16:07.111171998 +0000 UTC m=+0.157062146 container create c90ba430d17ae76bde41b57d0192df846669d4eefb6d3e6e66fdefa8a5f31727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T10:16:07.342 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:07 smithi047 podman[72067]: 2024-09-10 10:16:07.313707878 +0000 UTC m=+0.359598038 container init c90ba430d17ae76bde41b57d0192df846669d4eefb6d3e6e66fdefa8a5f31727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T10:16:07.342 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:07 smithi047 podman[72067]: 2024-09-10 10:16:07.317016042 +0000 UTC m=+0.362906197 container start c90ba430d17ae76bde41b57d0192df846669d4eefb6d3e6e66fdefa8a5f31727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T10:16:07.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:07.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:07.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:07.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:07.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:07.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:07.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:07.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:07.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:07 smithi047 podman[72067]: 2024-09-10 10:16:07.397076589 +0000 UTC m=+0.442966741 container attach c90ba430d17ae76bde41b57d0192df846669d4eefb6d3e6e66fdefa8a5f31727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:16:07.660 INFO:journalctl@ceph.osd.7.smithi047.stdout:Sep 10 10:16:07 smithi047 podman[72067]: 2024-09-10 10:16:07.508175655 +0000 UTC m=+0.554065801 container died c90ba430d17ae76bde41b57d0192df846669d4eefb6d3e6e66fdefa8a5f31727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:16:07.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:07.340+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:07.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:07.340+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:07.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:07.340+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:07.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:07.340+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:07.661 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:07.340+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:08.103 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T10:16:08.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:08.150 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T10:16:08.150 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T10:16:08.150 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.8 2024-09-10T10:16:08.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:07.662+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:08.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:07.662+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:08.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:07.662+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:08.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:07 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:07.662+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:08.151 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:07.790+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:08.151 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:07.790+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:08.152 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:07.790+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:08.152 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:07 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:07.790+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:08.453 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 systemd[1]: Stopping Ceph osd.8 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:08.453 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:08.336+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:08.453 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:08.336+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:08.454 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:08.336+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:08.454 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:08.336+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:08.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:08 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:08.345+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:08.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:08 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:08.345+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:08.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:08 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:08.345+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:08.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:08 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:08.345+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:08.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:08 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:08.345+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:08.786 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:08.452+0000 7f0c4b21d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:08.786 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:08.452+0000 7f0c4b21d640 -1 osd.8 98 *** Got signal Terminated *** 2024-09-10T10:16:08.786 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:08.452+0000 7f0c4b21d640 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:16:08.786 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:08.785+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:08.786 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:08.785+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:08.787 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:08.785+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:08.787 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:08 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:08.785+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:09.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:08.654+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:09.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:08.654+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:09.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:08.654+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:09.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:08 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:08.654+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:09.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:09.288+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:09.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:09.288+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:09.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:09.289+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:09.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:09.289+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:09.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:09.299+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:09.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:09.299+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:09.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:09.299+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:09.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:09.299+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:09.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:09 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:09.299+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:09.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:09.647+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:09.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:09.647+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:09.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:09.647+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:09.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:09 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:09.647+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:10.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:09.761+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:10.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:09.761+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:10.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:09.761+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:10.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:09 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:09.761+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:10.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:10.294+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:10.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:10.294+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:10.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:10.294+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:10.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:10.294+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:10.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:10.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:10.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:10.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:10.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:10.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:10.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:10.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:10.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:10.280+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:10.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:10.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:10.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:10.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:10.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:10.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:10.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:10 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:10.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:11.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:10.741+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:11.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:10.741+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:11.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:10.741+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:11.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:10 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:10.741+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:11.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:11.315+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:11.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:11.315+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:11.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:11.315+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:11.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:11.315+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:11.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:11 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:11.249+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:11.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:11 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:11.249+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:11.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:11 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:11.249+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:11.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:11 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:11.249+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:11.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:11 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:11.249+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:11.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:11.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:11.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:11.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:11.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:11.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:11.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:11.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:11.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:11 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:11.646+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:12.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:11.698+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:12.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:11.698+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:12.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:11.698+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:12.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:11 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:11.698+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:12.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:12.345+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:12.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:12.345+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:12.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:12.345+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:12.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:12.345+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:12.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:12.234+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:12.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:12.234+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:12.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:12.234+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:12.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:12.234+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:12.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:12 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:12.234+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:13.149 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:12.654+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.084313+0000 front 2024-09-10T10:15:16.084271+0000 (oldest deadline 2024-09-10T10:15:41.984420+0000) 2024-09-10T10:16:13.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:12.654+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:21.984826+0000 front 2024-09-10T10:15:21.984928+0000 (oldest deadline 2024-09-10T10:15:50.185375+0000) 2024-09-10T10:16:13.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:12.654+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:32.486265+0000 front 2024-09-10T10:15:32.486276+0000 (oldest deadline 2024-09-10T10:15:55.986208+0000) 2024-09-10T10:16:13.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:12 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8[52975]: 2024-09-10T10:16:12.654+0000 7f0c47822640 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:37.687295+0000 front 2024-09-10T10:15:37.687158+0000 (oldest deadline 2024-09-10T10:16:03.587066+0000) 2024-09-10T10:16:13.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:12.660+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:13.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:12.660+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:13.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:12.660+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:13.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:12.660+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:13.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:12 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:12.660+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:13.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:13 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:13.335+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:13.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:13 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:13.335+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:13.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:13 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:13.335+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:13.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:13 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:13.335+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:13.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:13 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:13.335+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:13.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:13 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:13.201+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:13.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:13 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:13.201+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:13.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:13 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:13.201+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:13.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:13 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:13.201+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:13.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:13 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:13.201+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:13.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:13.616+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:13.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:13.616+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:13.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:13.616+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:13.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:13.616+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:13.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:13 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:13.616+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:14.150 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:14 smithi162 podman[71038]: 2024-09-10 10:16:14.045319078 +0000 UTC m=+5.693987445 container died 2a159fc646f5dd284fcb744533eeb76fe6450c64abf4fcce66d976b3f77da64b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T10:16:14.649 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:14 smithi162 podman[71038]: 2024-09-10 10:16:14.426932487 +0000 UTC m=+6.075600855 container cleanup 2a159fc646f5dd284fcb744533eeb76fe6450c64abf4fcce66d976b3f77da64b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:16:14.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:14 smithi162 bash[71038]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8 2024-09-10T10:16:14.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:14 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:14.339+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:14.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:14 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:14.339+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:14.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:14 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:14.339+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:14.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:14 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:14.339+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:14.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:14 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:14.339+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:14.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:14 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:14.204+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:14.659 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:14 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:14.204+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:14.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:14 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:14.204+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:14.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:14 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:14.204+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:14.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:14 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:14.204+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:14.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:14 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:14.204+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:14.988 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:14 smithi162 podman[71056]: 2024-09-10 10:16:14.669760894 +0000 UTC m=+0.627476891 container remove 2a159fc646f5dd284fcb744533eeb76fe6450c64abf4fcce66d976b3f77da64b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-09-10T10:16:15.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:14 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:14.664+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:15.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:14 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:14.664+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:15.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:14 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:14.664+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:15.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:14 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:14.664+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:15.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:14 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:14.664+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:15.307 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:14 smithi162 podman[71142]: 2024-09-10 10:16:14.890634554 +0000 UTC m=+0.016005005 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:16:15.307 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:14 smithi162 podman[71142]: 2024-09-10 10:16:14.995627544 +0000 UTC m=+0.120998026 container create 9ec1ca8dab67e1ccdc1567758d6ddeb92b001c151c31efb23f23f65ec61f4603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T10:16:15.307 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:15 smithi162 podman[71142]: 2024-09-10 10:16:15.20611483 +0000 UTC m=+0.331485271 container init 9ec1ca8dab67e1ccdc1567758d6ddeb92b001c151c31efb23f23f65ec61f4603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:16:15.307 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:15 smithi162 podman[71142]: 2024-09-10 10:16:15.209512453 +0000 UTC m=+0.334882896 container start 9ec1ca8dab67e1ccdc1567758d6ddeb92b001c151c31efb23f23f65ec61f4603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T10:16:15.307 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:15 smithi162 podman[71142]: 2024-09-10 10:16:15.30637424 +0000 UTC m=+0.431744701 container attach 9ec1ca8dab67e1ccdc1567758d6ddeb92b001c151c31efb23f23f65ec61f4603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T10:16:15.409 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:15 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:15.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:15.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:15 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:15.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:15.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:15 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:15.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:15.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:15 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:15.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:15.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:15 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:15.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:15.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:15 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:15.154+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:15.650 INFO:journalctl@ceph.osd.8.smithi162.stdout:Sep 10 10:16:15 smithi162 podman[71142]: 2024-09-10 10:16:15.402411064 +0000 UTC m=+0.527781505 container died 9ec1ca8dab67e1ccdc1567758d6ddeb92b001c151c31efb23f23f65ec61f4603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:16:15.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:15 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:15.351+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:15.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:15 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:15.351+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:15.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:15 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:15.351+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:15.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:15 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:15.351+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:15.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:15 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:15.351+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:15.981 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T10:16:16.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:16.023 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T10:16:16.024 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T10:16:16.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.9 2024-09-10T10:16:16.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:15 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:15.707+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:16.052 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:15 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:15.707+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:16.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:15 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:15.707+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:16.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:15 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:15.707+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:16.053 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:15 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:15.707+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:16.330 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 systemd[1]: Stopping Ceph osd.9 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:16.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:16 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:16.395+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:16.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:16 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:16.395+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:16.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:16 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:16.395+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:16.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:16 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:16.395+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:16.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:16 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:16.395+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:16.651 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.328+0000 7f96e522c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:16.651 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.328+0000 7f96e522c640 -1 osd.9 98 *** Got signal Terminated *** 2024-09-10T10:16:16.651 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.328+0000 7f96e522c640 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:16:16.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:16 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:16.165+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:16.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:16 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:16.165+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:16.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:16 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:16.165+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:16.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:16 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:16.165+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:16.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:16 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:16.165+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:16.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:16 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:16.165+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:17.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.691+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:17.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.691+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:17.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.691+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:17.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.691+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:17.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:16 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:16.691+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:17.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:17 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:17.360+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:17.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:17 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:17.360+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:17.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:17 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:17.360+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:17.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:17 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:17.360+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:17.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:17 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:17.360+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:17.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:17 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:17.171+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:17.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:17 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:17.171+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:17.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:17 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:17.171+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:17.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:17 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:17.171+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:17.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:17 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:17.171+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:17.660 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:17 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:17.171+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:18.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:17 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:17.741+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:18.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:17 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:17.741+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:18.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:17 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:17.741+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:18.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:17 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:17.741+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:18.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:17 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:17.741+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:18.409 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:18 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:18.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:18.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:18 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:18.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:18.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:18 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:18.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:18.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:18 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:18.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:18.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:18 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:18.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:18.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:18 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:18.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:18.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:18 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:18.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:18.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:18 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:18.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:18.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:18 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:18.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:18.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:18 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:18.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:18.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:18 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:18.367+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:19.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:18 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:18.750+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:19.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:18 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:18.750+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:19.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:18 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:18.750+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:19.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:18 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:18.750+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:19.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:18 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:18.750+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:19.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:19.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:19.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:19.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:19.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:19.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:19.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:19.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:19.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:19.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:19.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:19 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:19.133+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:19.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:19 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:19.323+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:19.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:19 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:19.323+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:19.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:19 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:19.323+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:19.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:19 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:19.323+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:19.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:19 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:19.323+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:20.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:19.720+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:20.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:19.720+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:20.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:19.720+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:20.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:19.720+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:20.151 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:19 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:19.720+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:20.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:20 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:20.102+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:20.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:20 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:20.102+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:20.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:20 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:20.102+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:20.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:20 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:20.102+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:20.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:20 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:20.102+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:20.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:20 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:20.102+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:20.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:20 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:20.102+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:20.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:20.328+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:20.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:20.328+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:20.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:20.328+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:20.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:20.328+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:20.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:20.328+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:20.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:20 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:20.328+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:20.723+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:09.797527+0000 front 2024-09-10T10:15:09.797476+0000 (oldest deadline 2024-09-10T10:15:35.697554+0000) 2024-09-10T10:16:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:20.723+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.698084+0000 front 2024-09-10T10:15:15.698236+0000 (oldest deadline 2024-09-10T10:15:40.398220+0000) 2024-09-10T10:16:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:20.723+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.299635+0000 front 2024-09-10T10:15:23.299472+0000 (oldest deadline 2024-09-10T10:15:47.399475+0000) 2024-09-10T10:16:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:20.723+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:31.901349+0000 front 2024-09-10T10:15:31.901257+0000 (oldest deadline 2024-09-10T10:15:57.201349+0000) 2024-09-10T10:16:21.150 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:20 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9[65785]: 2024-09-10T10:16:20.723+0000 7f96e1831640 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:47.405018+0000 front 2024-09-10T10:15:47.404809+0000 (oldest deadline 2024-09-10T10:16:10.904810+0000) 2024-09-10T10:16:21.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:21 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:21.090+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:21.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:21 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:21.090+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:21.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:21 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:21.090+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:21.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:21 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:21.090+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:21.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:21 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:21.090+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:21.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:21 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:21.090+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:21.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:21 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:21.090+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:21.649 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:21 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:21.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:21.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:21 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:21.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:21.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:21 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:21.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:21.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:21 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:21.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:21.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:21 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:21.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:21.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:21 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:21.371+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:22.131 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:21 smithi045 podman[91478]: 2024-09-10 10:16:21.800263856 +0000 UTC m=+5.574336586 container died 75985b06322bdc7f0be2021f4717eba88f28d1c8762acd35e7e3d46161355f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T10:16:22.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:22 smithi045 podman[91478]: 2024-09-10 10:16:22.202987453 +0000 UTC m=+5.977060163 container cleanup 75985b06322bdc7f0be2021f4717eba88f28d1c8762acd35e7e3d46161355f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T10:16:22.434 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:22 smithi045 bash[91478]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9 2024-09-10T10:16:22.434 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:22 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:22.122+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:22.434 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:22 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:22.122+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:22.435 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:22 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:22.122+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:22.435 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:22 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:22.122+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:22.435 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:22 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:22.122+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:22.435 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:22 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:22.122+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:22.435 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:22 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:22.122+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:22.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:22 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:22.369+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:22.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:22 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:22.369+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:22.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:22 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:22.369+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:22.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:22 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:22.369+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:22.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:22 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:22.369+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:22.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:22 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:22.369+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:22.745 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:22 smithi045 podman[91493]: 2024-09-10 10:16:22.420932185 +0000 UTC m=+0.620770667 container remove 75985b06322bdc7f0be2021f4717eba88f28d1c8762acd35e7e3d46161355f78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T10:16:23.000 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:22 smithi045 podman[91585]: 2024-09-10 10:16:22.647719785 +0000 UTC m=+0.016289182 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:16:23.000 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:22 smithi045 podman[91585]: 2024-09-10 10:16:22.754743021 +0000 UTC m=+0.123312426 container create ca2d64a61e0eb920ccbb0e9a3310d35f7befbd1b72d2cffd7c3c811d92f3637f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:16:23.000 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:22 smithi045 podman[91585]: 2024-09-10 10:16:22.999121542 +0000 UTC m=+0.367690948 container init ca2d64a61e0eb920ccbb0e9a3310d35f7befbd1b72d2cffd7c3c811d92f3637f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T10:16:23.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:23 smithi045 podman[91585]: 2024-09-10 10:16:23.002929488 +0000 UTC m=+0.371498891 container start ca2d64a61e0eb920ccbb0e9a3310d35f7befbd1b72d2cffd7c3c811d92f3637f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:16:23.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:23 smithi045 podman[91585]: 2024-09-10 10:16:23.065261163 +0000 UTC m=+0.433830569 container attach ca2d64a61e0eb920ccbb0e9a3310d35f7befbd1b72d2cffd7c3c811d92f3637f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:16:23.400 INFO:journalctl@ceph.osd.9.smithi045.stdout:Sep 10 10:16:23 smithi045 podman[91585]: 2024-09-10 10:16:23.196933695 +0000 UTC m=+0.565503092 container died ca2d64a61e0eb920ccbb0e9a3310d35f7befbd1b72d2cffd7c3c811d92f3637f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:16:23.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:23.093+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:23.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:23.093+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:23.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:23.093+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:23.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:23.093+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:23.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:23.093+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:23.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:23.093+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:23.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:23.093+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:23.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T10:16:23.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:23.859 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T10:16:23.859 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T10:16:23.859 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.10 2024-09-10T10:16:23.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:23 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:23.419+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:23.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:23 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:23.419+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:23.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:23 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:23.419+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:23.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:23 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:23.419+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:23.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:23 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:23.419+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:23.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:23 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:23.419+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:23 smithi047 systemd[1]: Stopping Ceph osd.10 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.054+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.054+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.054+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.054+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.054+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.054+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:24.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.054+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:24.155 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.152+0000 7f9e2ee98640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:24.155 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.152+0000 7f9e2ee98640 -1 osd.10 98 *** Got signal Terminated *** 2024-09-10T10:16:24.155 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:24 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:24.152+0000 7f9e2ee98640 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:16:24.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:24 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:24.432+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:24.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:24 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:24.432+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:24.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:24 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:24.432+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:24.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:24 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:24.432+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:24.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:24 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:24.432+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:24.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:24 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:24.432+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:25.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:25 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:25.070+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:25.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:25 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:25.070+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:25.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:25 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:25.070+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:25.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:25 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:25.070+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:25.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:25 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:25.070+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:25.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:25 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:25.070+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:25.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:25 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:25.070+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:25.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:25.425+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:25.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:25.425+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:25.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:25.425+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:25.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:25.425+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:25.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:25.425+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:25.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:25.425+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:26.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:26 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:26.067+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:26.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:26 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:26.067+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:26.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:26 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:26.067+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:26.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:26 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:26.067+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:26.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:26 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:26.067+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:26.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:26 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:26.067+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:26.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:26 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:26.067+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:26.746 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:26 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:26.456+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:26.746 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:26 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:26.456+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:26.746 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:26 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:26.456+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:26.746 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:26 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:26.456+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:26.746 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:26 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:26.456+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:26.746 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:26 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:26.456+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:27.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:27 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:27.078+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:27.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:27 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:27.078+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:27.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:27 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:27.078+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:27.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:27 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:27.078+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:27.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:27 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:27.078+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:27.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:27 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:27.078+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:27.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:27 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:27.078+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:27.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:27 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:27.436+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:27.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:27 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:27.436+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:27.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:27 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:27.436+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:27.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:27 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:27.436+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:27.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:27 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:27.436+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:27.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:27 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:27.436+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:27.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:27 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:27.436+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:28.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:28 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:28.108+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:28.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:28 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:28.108+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:28.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:28 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:28.108+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:28.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:28 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:28.108+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:28.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:28 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:28.108+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:28.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:28 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:28.108+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:28.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:28 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:28.108+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:28.899 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:28 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:28.421+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:28.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:28 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:28.421+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:28.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:28 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:28.421+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:28.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:28 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:28.421+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:28.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:28 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:28.421+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:28.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:28 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:28.421+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:28.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:28 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:28.421+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:29.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:29.069+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:04.765349+0000 front 2024-09-10T10:15:04.765253+0000 (oldest deadline 2024-09-10T10:15:30.665086+0000) 2024-09-10T10:16:29.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:29.069+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:15.866510+0000 front 2024-09-10T10:15:15.866625+0000 (oldest deadline 2024-09-10T10:15:41.766766+0000) 2024-09-10T10:16:29.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:29.069+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:22.868140+0000 front 2024-09-10T10:15:22.868095+0000 (oldest deadline 2024-09-10T10:15:46.367960+0000) 2024-09-10T10:16:29.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:29.069+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-09-10T10:15:33.370072+0000 front 2024-09-10T10:15:33.369984+0000 (oldest deadline 2024-09-10T10:15:58.669756+0000) 2024-09-10T10:16:29.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:29.069+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:38.670048+0000 front 2024-09-10T10:15:38.670075+0000 (oldest deadline 2024-09-10T10:16:02.770079+0000) 2024-09-10T10:16:29.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:29.069+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:48.671130+0000 front 2024-09-10T10:15:48.670995+0000 (oldest deadline 2024-09-10T10:16:13.970853+0000) 2024-09-10T10:16:29.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10[60729]: 2024-09-10T10:16:29.069+0000 7f9e2b49d640 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:53.971523+0000 front 2024-09-10T10:15:53.971545+0000 (oldest deadline 2024-09-10T10:16:19.871305+0000) 2024-09-10T10:16:29.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:29.434+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:29.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:29.434+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:29.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:29.434+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:29.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:29.434+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:29.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:29.434+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:29.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:29.434+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:29.900 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:29 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:29.434+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:30.154 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:29 smithi047 podman[72172]: 2024-09-10 10:16:29.768184353 +0000 UTC m=+5.717262449 container died 143d2c57d1b289100c91da792a34636816232fc3f1fbc03532f87678148d5955 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T10:16:30.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:30 smithi047 podman[72172]: 2024-09-10 10:16:30.252296019 +0000 UTC m=+6.201374104 container cleanup 143d2c57d1b289100c91da792a34636816232fc3f1fbc03532f87678148d5955 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T10:16:30.410 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:30 smithi047 bash[72172]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10 2024-09-10T10:16:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:30.391+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:30.391+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:30.391+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:30.391+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:30.391+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:30.391+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:30.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:30.391+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:30.804 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:30 smithi047 podman[72187]: 2024-09-10 10:16:30.503562405 +0000 UTC m=+0.732258931 container remove 143d2c57d1b289100c91da792a34636816232fc3f1fbc03532f87678148d5955 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T10:16:31.082 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:30 smithi047 podman[72276]: 2024-09-10 10:16:30.804149141 +0000 UTC m=+0.093825699 container create 115d02ee747d0109eacd729bc143900b6a1a49998911b14c2483d000628ac121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:16:31.082 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:30 smithi047 podman[72276]: 2024-09-10 10:16:30.726155393 +0000 UTC m=+0.015831923 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:16:31.082 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:31 smithi047 podman[72276]: 2024-09-10 10:16:31.015298061 +0000 UTC m=+0.304974591 container init 115d02ee747d0109eacd729bc143900b6a1a49998911b14c2483d000628ac121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:16:31.082 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:31 smithi047 podman[72276]: 2024-09-10 10:16:31.018764578 +0000 UTC m=+0.308441099 container start 115d02ee747d0109eacd729bc143900b6a1a49998911b14c2483d000628ac121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:16:31.082 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:31 smithi047 podman[72276]: 2024-09-10 10:16:31.081480733 +0000 UTC m=+0.371157253 container attach 115d02ee747d0109eacd729bc143900b6a1a49998911b14c2483d000628ac121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T10:16:31.409 INFO:journalctl@ceph.osd.10.smithi047.stdout:Sep 10 10:16:31 smithi047 podman[72276]: 2024-09-10 10:16:31.208134632 +0000 UTC m=+0.497811164 container died 115d02ee747d0109eacd729bc143900b6a1a49998911b14c2483d000628ac121 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:16:31.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:31.382+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:31.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:31.382+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:31.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:31.382+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:31.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:31.382+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:31.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:31.382+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:31.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:31.382+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:31.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:31 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:31.382+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:31.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T10:16:31.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:31.965 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T10:16:31.965 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T10:16:31.966 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.11 2024-09-10T10:16:32.320 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 systemd[1]: Stopping Ceph osd.11 for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:32.320 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.318+0000 7ff701a62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:32.320 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.318+0000 7ff701a62640 -1 osd.11 98 *** Got signal Terminated *** 2024-09-10T10:16:32.320 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.318+0000 7ff701a62640 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:16:32.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:32.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:32.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:32.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:32.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:32.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:32.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:32 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:32.380+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:33.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:33.379+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:33.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:33.379+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:33.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:33.379+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:33.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:33.379+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:33.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:33.379+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:33.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:33.379+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:33.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:33 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:33.379+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:34.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:34.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:34.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:34.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:34.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:34.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:34.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:34.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:34.651 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:34.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:34.651 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:34.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:34.651 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:34 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:34.362+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:35.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:35 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:35.337+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-09-10T10:16:12.421532+0000 front 2024-09-10T10:16:12.421511+0000 (oldest deadline 2024-09-10T10:16:34.721537+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:36.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:36 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:36.310+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-09-10T10:16:12.421532+0000 front 2024-09-10T10:16:12.421511+0000 (oldest deadline 2024-09-10T10:16:34.721537+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-09-10T10:15:06.913547+0000 front 2024-09-10T10:15:06.913506+0000 (oldest deadline 2024-09-10T10:15:32.813448+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6804 osd.1 since back 2024-09-10T10:15:16.215129+0000 front 2024-09-10T10:15:16.215115+0000 (oldest deadline 2024-09-10T10:15:42.115022+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6804 osd.2 since back 2024-09-10T10:15:23.216068+0000 front 2024-09-10T10:15:23.216030+0000 (oldest deadline 2024-09-10T10:15:49.116087+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6812 osd.4 since back 2024-09-10T10:15:40.917598+0000 front 2024-09-10T10:15:40.917732+0000 (oldest deadline 2024-09-10T10:16:06.817473+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6812 osd.5 since back 2024-09-10T10:15:46.817755+0000 front 2024-09-10T10:15:46.817836+0000 (oldest deadline 2024-09-10T10:16:12.618319+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-09-10T10:15:54.919684+0000 front 2024-09-10T10:15:54.919549+0000 (oldest deadline 2024-09-10T10:16:20.219496+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.47:6820 osd.7 since back 2024-09-10T10:16:03.620750+0000 front 2024-09-10T10:16:03.620729+0000 (oldest deadline 2024-09-10T10:16:26.520670+0000) 2024-09-10T10:16:37.650 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11[59564]: 2024-09-10T10:16:37.271+0000 7ff6fa05f640 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.162:6820 osd.8 since back 2024-09-10T10:16:12.421532+0000 front 2024-09-10T10:16:12.421511+0000 (oldest deadline 2024-09-10T10:16:34.721537+0000) 2024-09-10T10:16:38.079 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:37 smithi162 podman[71258]: 2024-09-10 10:16:37.814642844 +0000 UTC m=+5.651566621 container died 69daefcab44f30fd0dc405b2953308138cae0b0cd07cb0974081d5fcda794baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T10:16:38.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:38 smithi162 podman[71258]: 2024-09-10 10:16:38.185454099 +0000 UTC m=+6.022377875 container cleanup 69daefcab44f30fd0dc405b2953308138cae0b0cd07cb0974081d5fcda794baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:16:38.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:38 smithi162 bash[71258]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11 2024-09-10T10:16:38.798 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:38 smithi162 podman[71287]: 2024-09-10 10:16:38.453503283 +0000 UTC m=+0.635981696 container remove 69daefcab44f30fd0dc405b2953308138cae0b0cd07cb0974081d5fcda794baf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:16:39.098 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:38 smithi162 podman[71377]: 2024-09-10 10:16:38.70079518 +0000 UTC m=+0.015972321 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:16:39.098 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:38 smithi162 podman[71377]: 2024-09-10 10:16:38.803979555 +0000 UTC m=+0.119156684 container create 6817436524ca0f9fe49f6e7f963d6201dba9b56b1dd5bddc6115a5d5be2972dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:16:39.098 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 podman[71377]: 2024-09-10 10:16:39.023055545 +0000 UTC m=+0.338232672 container init 6817436524ca0f9fe49f6e7f963d6201dba9b56b1dd5bddc6115a5d5be2972dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T10:16:39.098 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 podman[71377]: 2024-09-10 10:16:39.026352567 +0000 UTC m=+0.341529708 container start 6817436524ca0f9fe49f6e7f963d6201dba9b56b1dd5bddc6115a5d5be2972dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:16:39.098 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 podman[71377]: 2024-09-10 10:16:39.097908113 +0000 UTC m=+0.413085255 container attach 6817436524ca0f9fe49f6e7f963d6201dba9b56b1dd5bddc6115a5d5be2972dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T10:16:39.400 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 podman[71377]: 2024-09-10 10:16:39.221985257 +0000 UTC m=+0.537162398 container died 6817436524ca0f9fe49f6e7f963d6201dba9b56b1dd5bddc6115a5d5be2972dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:16:39.734 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T10:16:39.757 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 podman[71400]: 2024-09-10 10:16:39.718032765 +0000 UTC m=+0.490503607 container remove 6817436524ca0f9fe49f6e7f963d6201dba9b56b1dd5bddc6115a5d5be2972dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T10:16:39.757 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-10T10:16:39.758 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 systemd[1]: Stopped Ceph osd.11 for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:16:39.758 INFO:journalctl@ceph.osd.11.smithi162.stdout:Sep 10 10:16:39 smithi162 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@osd.11.service: Consumed 5min 27.831s CPU time. 2024-09-10T10:16:39.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:39.772 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T10:16:39.772 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T10:16:39.773 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.a 2024-09-10T10:16:40.074 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:16:39 smithi045 systemd[1]: Stopping Ceph mds.a for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:40.400 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:16:40 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a[73022]: 2024-09-10T10:16:40.072+0000 7f194760f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:40.400 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:16:40 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a[73022]: 2024-09-10T10:16:40.072+0000 7f194760f640 -1 mds.a *** got signal Terminated *** 2024-09-10T10:16:42.150 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:16:41 smithi045 podman[91705]: 2024-09-10 10:16:41.899205311 +0000 UTC m=+1.928750050 container died ff9749f130a3f8107d93e6345925616f2d445397076c1c7f6b056e44aa1c99a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T10:16:42.525 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:16:42 smithi045 podman[91705]: 2024-09-10 10:16:42.248520316 +0000 UTC m=+2.278065099 container cleanup ff9749f130a3f8107d93e6345925616f2d445397076c1c7f6b056e44aa1c99a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T10:16:42.525 INFO:journalctl@ceph.mds.a.smithi045.stdout:Sep 10 10:16:42 smithi045 bash[91705]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-a 2024-09-10T10:16:42.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T10:16:42.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:42.722 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T10:16:42.723 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T10:16:42.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.d 2024-09-10T10:16:43.045 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 10 10:16:42 smithi045 systemd[1]: Stopping Ceph mds.d for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:43.400 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 10 10:16:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-d[71911]: 2024-09-10T10:16:43.044+0000 7f0a9ab89640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:43.400 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 10 10:16:43 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-d[71911]: 2024-09-10T10:16:43.044+0000 7f0a9ab89640 -1 mds.d *** got signal Terminated *** 2024-09-10T10:16:47.328 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 10 10:16:47 smithi045 podman[91826]: 2024-09-10 10:16:47.056007299 +0000 UTC m=+4.106590392 container died acb83917740106b725ec085f823d3b097f400c0ea46e6a28de95e90e78096230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:16:47.650 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 10 10:16:47 smithi045 podman[91826]: 2024-09-10 10:16:47.544043831 +0000 UTC m=+4.594626923 container cleanup acb83917740106b725ec085f823d3b097f400c0ea46e6a28de95e90e78096230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T10:16:47.650 INFO:journalctl@ceph.mds.d.smithi045.stdout:Sep 10 10:16:47 smithi045 bash[91826]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-d 2024-09-10T10:16:47.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T10:16:48.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:48.022 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T10:16:48.022 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T10:16:48.022 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.g 2024-09-10T10:16:48.357 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:16:48 smithi045 systemd[1]: Stopping Ceph mds.g for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:48.650 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:16:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-g[72690]: 2024-09-10T10:16:48.355+0000 7f1d55d55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:48.650 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:16:48 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-g[72690]: 2024-09-10T10:16:48.355+0000 7f1d55d55640 -1 mds.g *** got signal Terminated *** 2024-09-10T10:16:51.747 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:16:51 smithi045 podman[91946]: 2024-09-10 10:16:51.475292571 +0000 UTC m=+3.223809475 container died f0e788d91c92f4e0753dee441844b85db49b2d7561bb68dce0c2011163c1830b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T10:16:52.089 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:16:51 smithi045 podman[91946]: 2024-09-10 10:16:51.845752756 +0000 UTC m=+3.594269699 container cleanup f0e788d91c92f4e0753dee441844b85db49b2d7561bb68dce0c2011163c1830b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-g, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:16:52.089 INFO:journalctl@ceph.mds.g.smithi045.stdout:Sep 10 10:16:51 smithi045 bash[91946]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-g 2024-09-10T10:16:52.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T10:16:52.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:52.272 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T10:16:52.272 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T10:16:52.272 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.j 2024-09-10T10:16:52.650 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:16:52 smithi045 systemd[1]: Stopping Ceph mds.j for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:53.150 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:16:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j[71491]: 2024-09-10T10:16:52.699+0000 7f261db89640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:53.150 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:16:52 smithi045 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j[71491]: 2024-09-10T10:16:52.699+0000 7f261db89640 -1 mds.j *** got signal Terminated *** 2024-09-10T10:16:57.032 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:16:56 smithi045 podman[92070]: 2024-09-10 10:16:56.713475835 +0000 UTC m=+4.218946000 container died bf77fe6945122a2f0c7029404d54a0a22a5733e58ac9a11daa2e5755f36392e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T10:16:57.033 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:16:57 smithi045 podman[92070]: 2024-09-10 10:16:57.032088319 +0000 UTC m=+4.537558474 container cleanup bf77fe6945122a2f0c7029404d54a0a22a5733e58ac9a11daa2e5755f36392e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T10:16:57.400 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:16:57 smithi045 bash[92070]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j 2024-09-10T10:16:57.400 INFO:journalctl@ceph.mds.j.smithi045.stdout:Sep 10 10:16:57 smithi045 podman[92081]: 2024-09-10 10:16:57.217005322 +0000 UTC m=+0.499675629 container remove bf77fe6945122a2f0c7029404d54a0a22a5733e58ac9a11daa2e5755f36392e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T10:16:57.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T10:16:57.496 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:16:57.497 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T10:16:57.497 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T10:16:57.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.b 2024-09-10T10:16:57.821 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:16:57 smithi047 systemd[1]: Stopping Ceph mds.b for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:16:58.159 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:16:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b[66141]: 2024-09-10T10:16:57.818+0000 7f02f76ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:16:58.160 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:16:57 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b[66141]: 2024-09-10T10:16:57.818+0000 7f02f76ce640 -1 mds.b *** got signal Terminated *** 2024-09-10T10:17:02.095 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:17:01 smithi047 podman[72390]: 2024-09-10 10:17:01.824519563 +0000 UTC m=+4.134694352 container died f6d7311f537f63baa6e95a8e2ab1b23ec29ceeac20f123c97ee90081b4228c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T10:17:02.387 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:17:02 smithi047 podman[72390]: 2024-09-10 10:17:02.177516409 +0000 UTC m=+4.487691197 container cleanup f6d7311f537f63baa6e95a8e2ab1b23ec29ceeac20f123c97ee90081b4228c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T10:17:02.388 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:17:02 smithi047 bash[72390]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b 2024-09-10T10:17:02.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T10:17:02.639 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:17:02 smithi047 podman[72402]: 2024-09-10 10:17:02.387010855 +0000 UTC m=+0.551421030 container remove f6d7311f537f63baa6e95a8e2ab1b23ec29ceeac20f123c97ee90081b4228c30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T10:17:02.639 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:17:02 smithi047 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-10T10:17:02.639 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:17:02 smithi047 systemd[1]: Stopped Ceph mds.b for 729b10fe-6f5a-11ef-bcea-c7b262605968. 2024-09-10T10:17:02.639 INFO:journalctl@ceph.mds.b.smithi047.stdout:Sep 10 10:17:02 smithi047 systemd[1]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.b.service: Consumed 7min 32.415s CPU time. 2024-09-10T10:17:02.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:02.651 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T10:17:02.651 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T10:17:02.652 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.e 2024-09-10T10:17:02.909 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:17:02 smithi047 systemd[1]: Stopping Ceph mds.e for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:17:03.409 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:17:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e[65533]: 2024-09-10T10:17:03.089+0000 7fd4311b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:17:03.410 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:17:03 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e[65533]: 2024-09-10T10:17:03.089+0000 7fd4311b0640 -1 mds.e *** got signal Terminated *** 2024-09-10T10:17:06.625 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:17:06 smithi047 podman[72510]: 2024-09-10 10:17:06.330549068 +0000 UTC m=+3.450062532 container died 0ca7f833c8d3ac44d6346d66270300dae12f5d0e1c651edaf670af3b8d53aaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T10:17:06.909 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:17:06 smithi047 podman[72510]: 2024-09-10 10:17:06.732037047 +0000 UTC m=+3.851550515 container cleanup 0ca7f833c8d3ac44d6346d66270300dae12f5d0e1c651edaf670af3b8d53aaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:17:06.910 INFO:journalctl@ceph.mds.e.smithi047.stdout:Sep 10 10:17:06 smithi047 bash[72510]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-e 2024-09-10T10:17:07.114 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T10:17:07.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:07.147 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T10:17:07.147 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T10:17:07.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.h 2024-09-10T10:17:07.510 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:17:07 smithi047 systemd[1]: Stopping Ceph mds.h for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:17:07.909 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:17:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h[65838]: 2024-09-10T10:17:07.509+0000 7f4de3cae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:17:07.910 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:17:07 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h[65838]: 2024-09-10T10:17:07.509+0000 7f4de3cae640 -1 mds.h *** got signal Terminated *** 2024-09-10T10:17:09.909 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:17:09 smithi047 podman[72631]: 2024-09-10 10:17:09.632031372 +0000 UTC m=+2.258026577 container died 5548595c5848fd007ae326a8a25ff42a62b941142530631296e3fe9130d31965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T10:17:10.380 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:17:09 smithi047 podman[72631]: 2024-09-10 10:17:09.959984868 +0000 UTC m=+2.585980071 container cleanup 5548595c5848fd007ae326a8a25ff42a62b941142530631296e3fe9130d31965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T10:17:10.380 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:17:09 smithi047 bash[72631]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h 2024-09-10T10:17:10.380 INFO:journalctl@ceph.mds.h.smithi047.stdout:Sep 10 10:17:10 smithi047 podman[72645]: 2024-09-10 10:17:10.136109655 +0000 UTC m=+0.503691161 container remove 5548595c5848fd007ae326a8a25ff42a62b941142530631296e3fe9130d31965 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:17:10.397 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T10:17:10.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:10.430 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T10:17:10.430 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T10:17:10.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.k 2024-09-10T10:17:10.659 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:17:10 smithi047 systemd[1]: Stopping Ceph mds.k for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:17:11.159 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:17:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k[65203]: 2024-09-10T10:17:10.746+0000 7fc2143ca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:17:11.160 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:17:10 smithi047 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k[65203]: 2024-09-10T10:17:10.746+0000 7fc2143ca640 -1 mds.k *** got signal Terminated *** 2024-09-10T10:17:13.909 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:17:13 smithi047 podman[72751]: 2024-09-10 10:17:13.654332299 +0000 UTC m=+3.006590978 container died 1b558e67b92b6e7a5aa9e3da24299432c7c88ff4af97440e99cf665fbacfa305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T10:17:14.353 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:17:13 smithi047 podman[72751]: 2024-09-10 10:17:13.996670801 +0000 UTC m=+3.348929460 container cleanup 1b558e67b92b6e7a5aa9e3da24299432c7c88ff4af97440e99cf665fbacfa305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T10:17:14.353 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:17:13 smithi047 bash[72751]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k 2024-09-10T10:17:14.353 INFO:journalctl@ceph.mds.k.smithi047.stdout:Sep 10 10:17:14 smithi047 podman[72764]: 2024-09-10 10:17:14.155801997 +0000 UTC m=+0.496214962 container remove 1b558e67b92b6e7a5aa9e3da24299432c7c88ff4af97440e99cf665fbacfa305 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:17:14.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T10:17:14.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:14.399 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T10:17:14.399 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T10:17:14.399 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.c 2024-09-10T10:17:14.716 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:17:14 smithi162 systemd[1]: Stopping Ceph mds.c for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:17:14.968 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:17:14 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c[64441]: 2024-09-10T10:17:14.714+0000 7fe05069c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:17:14.968 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:17:14 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c[64441]: 2024-09-10T10:17:14.714+0000 7fe05069c640 -1 mds.c *** got signal Terminated *** 2024-09-10T10:17:18.998 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:17:18 smithi162 podman[71479]: 2024-09-10 10:17:18.731288055 +0000 UTC m=+4.142286875 container died b1281eee2588ee97af4b46cafc47f7a77727dccd5fc9dd8c85039b513d2e9dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T10:17:19.349 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:17:19 smithi162 podman[71479]: 2024-09-10 10:17:19.089322076 +0000 UTC m=+4.500320884 container cleanup b1281eee2588ee97af4b46cafc47f7a77727dccd5fc9dd8c85039b513d2e9dbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T10:17:19.349 INFO:journalctl@ceph.mds.c.smithi162.stdout:Sep 10 10:17:19 smithi162 bash[71479]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-c 2024-09-10T10:17:19.521 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T10:17:19.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:19.559 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T10:17:19.559 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T10:17:19.559 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.f 2024-09-10T10:17:19.877 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:17:19 smithi162 systemd[1]: Stopping Ceph mds.f for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:17:20.149 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:17:19 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f[64142]: 2024-09-10T10:17:19.875+0000 7f6ba8741640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:17:20.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:17:19 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f[64142]: 2024-09-10T10:17:19.875+0000 7f6ba8741640 -1 mds.f *** got signal Terminated *** 2024-09-10T10:17:24.150 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:17:23 smithi162 podman[71613]: 2024-09-10 10:17:23.882788644 +0000 UTC m=+4.108639753 container died 55f6f8d1ef32dfd575112bb8fc857ee7fc8f0173c13ba9c22474b24a7173045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:17:24.611 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:17:24 smithi162 podman[71613]: 2024-09-10 10:17:24.392661624 +0000 UTC m=+4.618512733 container cleanup 55f6f8d1ef32dfd575112bb8fc857ee7fc8f0173c13ba9c22474b24a7173045b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T10:17:24.611 INFO:journalctl@ceph.mds.f.smithi162.stdout:Sep 10 10:17:24 smithi162 bash[71613]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-f 2024-09-10T10:17:24.753 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T10:17:24.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:24.787 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T10:17:24.787 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T10:17:24.788 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.i 2024-09-10T10:17:25.149 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:17:24 smithi162 systemd[1]: Stopping Ceph mds.i for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:17:25.149 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:17:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i[64742]: 2024-09-10T10:17:25.105+0000 7f36e25c8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:17:25.150 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:17:25 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i[64742]: 2024-09-10T10:17:25.105+0000 7f36e25c8640 -1 mds.i *** got signal Terminated *** 2024-09-10T10:17:29.400 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:17:29 smithi162 podman[71737]: 2024-09-10 10:17:29.116705332 +0000 UTC m=+4.117771998 container died 8648587df7092205515ebbf159735fc8d1380bf80b176151e6d0cc86d62d42c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T10:17:29.847 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:17:29 smithi162 podman[71737]: 2024-09-10 10:17:29.496001189 +0000 UTC m=+4.497067855 container cleanup 8648587df7092205515ebbf159735fc8d1380bf80b176151e6d0cc86d62d42c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:17:29.847 INFO:journalctl@ceph.mds.i.smithi162.stdout:Sep 10 10:17:29 smithi162 bash[71737]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-i 2024-09-10T10:17:30.074 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T10:17:30.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:30.106 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T10:17:30.107 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T10:17:30.107 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.l 2024-09-10T10:17:30.399 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:17:30 smithi162 systemd[1]: Stopping Ceph mds.l for 729b10fe-6f5a-11ef-bcea-c7b262605968... 2024-09-10T10:17:30.899 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:17:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l[63827]: 2024-09-10T10:17:30.424+0000 7f308d64c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:17:30.900 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:17:30 smithi162 ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l[63827]: 2024-09-10T10:17:30.424+0000 7f308d64c640 -1 mds.l *** got signal Terminated *** 2024-09-10T10:17:34.718 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:17:34 smithi162 podman[71860]: 2024-09-10 10:17:34.431008343 +0000 UTC m=+4.106497228 container died 5564991b156ee7bcdc93fb920dd283bf72f66ed11bc1d5e0812608c9c4aea0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T10:17:35.059 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:17:34 smithi162 podman[71860]: 2024-09-10 10:17:34.865942773 +0000 UTC m=+4.541431659 container cleanup 5564991b156ee7bcdc93fb920dd283bf72f66ed11bc1d5e0812608c9c4aea0f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:17:35.059 INFO:journalctl@ceph.mds.l.smithi162.stdout:Sep 10 10:17:34 smithi162 bash[71860]: ceph-729b10fe-6f5a-11ef-bcea-c7b262605968-mds-l 2024-09-10T10:17:35.277 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-729b10fe-6f5a-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T10:17:35.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:17:35.309 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T10:17:35.309 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:17:35.520 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T10:17:38.136 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:17:38.343 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T10:17:40.816 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:17:41.016 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T10:17:43.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:17:43.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:17:43.496 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:17:43.523 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T10:17:43.525 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi045/crash 2024-09-10T10:17:43.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:17:43.555 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:17:43.556 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:17:43.557 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi047/crash 2024-09-10T10:17:43.558 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:17:43.618 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:17:43.618 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:17:43.620 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi162/crash 2024-09-10T10:17:43.621 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:17:43.660 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:17:43.660 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:17:43.662 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T10:17:43.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-10T10:17:43.712 INFO:tasks.cephadm:Compressing logs... 2024-09-10T10:17:43.712 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:17:43.715 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:17:43.718 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:17:43.739 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T10:17:43.740 INFO:teuthology.orchestra.run.smithi045.stderr:: No such file or directory 2024-09-10T10:17:43.740 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T10:17:43.741 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:17:43.741 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:17:43.741 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T10:17:43.741 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:17:43.742 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T10:17:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T10:17:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.7% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:17:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T10:17:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.g.log: --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:17:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:17:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T10:17:43.754 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log: 94.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T10:17:43.754 INFO:teuthology.orchestra.run.smithi045.stderr: 94.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T10:17:43.754 INFO:teuthology.orchestra.run.smithi045.stderr: 86.5% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:17:43.754 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T10:17:43.754 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T10:17:43.754 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T10:17:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T10:17:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:17:43.758 INFO:teuthology.orchestra.run.smithi045.stderr: 94.6% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T10:17:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.3.log: 90.9% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:: No such file or directory 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.i.log/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log: 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.11.log: 82.1% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T10:17:43.760 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr: 90.6% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.l.log: 86.5% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.8.log: 94.5% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.z.log: 91.0% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:17:43.761 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T10:17:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:: No such file or directory 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log: 90.6% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T10:17:43.763 INFO:teuthology.orchestra.run.smithi047.stderr: 94.5% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T10:17:43.764 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.1% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:17:43.764 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:17:43.764 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T10:17:43.764 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log: 86.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:17:43.764 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T10:17:43.764 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.h.log: 90.9% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:17:43.767 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.c.log: 91.9% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T10:17:43.767 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.f.log: 94.5% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi162.stderr: 94.6% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi047.stderr: 94.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 91.8% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:17:43.768 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:17:43.770 INFO:teuthology.orchestra.run.smithi047.stderr: 94.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T10:17:43.779 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: 93.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:17:44.028 INFO:teuthology.orchestra.run.smithi162.stderr: 91.8% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T10:17:44.117 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T10:17:44.214 INFO:teuthology.orchestra.run.smithi047.stderr: 91.8% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T10:17:44.652 INFO:teuthology.orchestra.run.smithi045.stderr: 90.3% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T10:17:48.035 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T10:17:48.679 INFO:teuthology.orchestra.run.smithi162.stderr: 93.2% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T10:17:49.291 INFO:teuthology.orchestra.run.smithi045.stderr: 93.3% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T10:17:49.357 INFO:teuthology.orchestra.run.smithi047.stderr: 93.2% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T10:17:49.872 INFO:teuthology.orchestra.run.smithi047.stderr: 93.3% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T10:17:50.092 INFO:teuthology.orchestra.run.smithi162.stderr: 93.1% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T10:17:50.311 INFO:teuthology.orchestra.run.smithi162.stderr: 93.1% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T10:17:50.572 INFO:teuthology.orchestra.run.smithi162.stderr: 93.1% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T10:17:50.960 INFO:teuthology.orchestra.run.smithi045.stderr: 93.2% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T10:17:51.294 INFO:teuthology.orchestra.run.smithi045.stderr: 93.2% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T10:17:51.296 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-09-10T10:17:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m7.567s 2024-09-10T10:17:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m25.052s 2024-09-10T10:17:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.856s 2024-09-10T10:17:51.337 INFO:teuthology.orchestra.run.smithi047.stderr: 93.2% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T10:17:51.486 INFO:teuthology.orchestra.run.smithi047.stderr: 93.1% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T10:17:55.206 INFO:teuthology.orchestra.run.smithi162.stderr: 97.5% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T10:17:55.208 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-10T10:17:55.208 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m11.475s 2024-09-10T10:17:55.209 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m35.031s 2024-09-10T10:17:55.209 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m1.385s 2024-09-10T10:19:49.872 INFO:teuthology.orchestra.run.smithi047.stderr: 94.2% -- replaced with /var/log/ceph/729b10fe-6f5a-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T10:19:49.873 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-09-10T10:19:49.873 INFO:teuthology.orchestra.run.smithi047.stderr:real 2m6.140s 2024-09-10T10:19:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:user 2m28.701s 2024-09-10T10:19:49.874 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m4.679s 2024-09-10T10:19:49.874 INFO:tasks.cephadm:Archiving logs... 2024-09-10T10:19:49.875 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi045/log 2024-09-10T10:19:49.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:19:53.457 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi047/log 2024-09-10T10:19:53.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:20:13.731 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi162/log 2024-09-10T10:20:13.732 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:20:18.005 INFO:tasks.cephadm:Removing cluster... 2024-09-10T10:20:18.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 --force 2024-09-10T10:20:18.219 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T10:20:18.518 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 --force 2024-09-10T10:20:18.726 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T10:20:19.183 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 729b10fe-6f5a-11ef-bcea-c7b262605968 --force 2024-09-10T10:20:19.385 INFO:teuthology.orchestra.run.smithi162.stdout:Deleting cluster with fsid: 729b10fe-6f5a-11ef-bcea-c7b262605968 2024-09-10T10:20:19.729 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T10:20:19.730 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:20:19.749 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:20:19.767 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:20:19.783 INFO:tasks.cephadm:Teardown complete 2024-09-10T10:20:19.783 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T10:20:19.796 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T10:20:19.797 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:20:19.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:20:19.809 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:20:19.923 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T10:20:19.933 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T10:20:19.933 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:20:19.936 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:20:19.938 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:20:19.951 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:20:19.952 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:20:19.952 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:20:19.956 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:20:19.956 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-09-10T10:20:19.956 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -360us[ -350us] +/- 42ms 2024-09-10T10:20:19.956 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 +428us[ +428us] +/- 56ms 2024-09-10T10:20:19.956 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +183us[ +183us] +/- 69ms 2024-09-10T10:20:19.957 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:20:19.958 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:20:19.958 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-10T10:20:19.958 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 -338us[ -331us] +/- 42ms 2024-09-10T10:20:19.958 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +433us[ +433us] +/- 56ms 2024-09-10T10:20:19.958 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +157us[ +165us] +/- 69ms 2024-09-10T10:20:19.958 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:20:19.959 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:20:19.959 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-09-10T10:20:19.959 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 -348us[ -312us] +/- 42ms 2024-09-10T10:20:19.959 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 +428us[ +464us] +/- 56ms 2024-09-10T10:20:19.959 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +170us[ +206us] +/- 69ms 2024-09-10T10:20:19.959 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:20:19.959 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T10:20:19.969 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T10:20:19.970 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T10:20:19.997 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:20:20.080 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:20:20.153 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:20:20.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:20:20.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:20:20.251 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:20:20.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:20:20.281 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:20:20.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:20:20.310 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T10:20:20.321 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T10:20:20.332 INFO:teuthology.task.internal:Duration was 2012.374095 seconds 2024-09-10T10:20:20.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T10:20:20.343 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T10:20:20.344 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:20:20.346 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:20:20.349 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:20:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:20:20.387 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:20:20.389 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:20:20.701 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T10:20:20.704 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-09-10T10:20:20.704 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:20:20.726 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-09-10T10:20:20.727 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:20:20.769 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-09-10T10:20:20.769 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:20:20.798 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T10:20:20.798 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:20:20.801 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:20:20.812 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:20:20.865 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T10:20:20.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:20:20.868 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:20:20.881 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:20:21.326 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T10:20:21.338 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T10:20:21.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:20:21.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:20:21.392 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:20:21.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T10:20:21.625 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:20:21.628 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:20:21.631 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:20:21.650 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-09-10T10:20:21.652 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-09-10T10:20:21.654 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-09-10T10:20:21.681 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:20:21.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:20:21.720 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:20:21.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:20:21.738 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:20:21.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:20:21.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T10:20:21.764 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T10:20:21.766 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi045 2024-09-10T10:20:21.766 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:20:21.811 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi047 2024-09-10T10:20:21.811 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:20:21.847 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899009/remote/smithi162 2024-09-10T10:20:21.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:20:21.885 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T10:20:21.885 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:20:21.887 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:20:21.889 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:20:21.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T10:20:21.947 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T10:20:21.947 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T10:20:21.961 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T10:20:21.961 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:20:21.964 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:20:21.966 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:20:21.977 INFO:teuthology.orchestra.run.smithi045.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:20 /home/ubuntu/cephtest 2024-09-10T10:20:21.978 INFO:teuthology.orchestra.run.smithi047.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:20 /home/ubuntu/cephtest 2024-09-10T10:20:21.988 INFO:teuthology.orchestra.run.smithi162.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:20 /home/ubuntu/cephtest 2024-09-10T10:20:21.989 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T10:20:22.019 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T10:20:22.134 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} duration: 2012.3740952014923 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T10:20:22.134 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:20:22.225 INFO:teuthology.run:pass